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This program refuiew^has been. prepared usi*ng' the -criteria dteW 

• ^termijied Educatibnal Management j>rogram Director Rich^i-'d " % 

^Watlq^ns. Other criteria .v^ged by the' feVidWer inclGaed those o'f 
-relevancy, feasibiMty and utility (o'peptional definitions of 

-tl^^se^ terms will be supplied later 'in the report). A "miscella- 
neous" .section 'is included in 'the report covering other thoughts 

/which occurred^ to the iread^r but- which did nfot match up vith the • 

/ ' « ' ^ ' . " ' • . ; ^ 

other criteria. . ' 

The "Designing- Instructional Prpgrams" ;>publication, "Curri-- 
culum Analyzer," was used to summarize and cpnderxse certain 

features of the curriculum; and the procedure o^^t lined on page IX H 

» J. 
of the publication was foll,owed tg arrive at the\ rating scores. ' ^ 

Each of the training bo'oklets was returned with the re- ' 

• viewer s comments. The booklets also contained* various comments. 

■ ' . ' ' ■ J- . ^ 

.The first section of the. report addresses the questions raised 

by the rejpreser.tative from the Far West Laboratory; the second 

"* '•*. ^ - ' '• ' • i. 

section, in addM^on to'thos'e above, addrfsse? /criteria prepared 

' •'•/'■ • ^ ■ ' " - ' ■ - ■ 

by the reviewer^; and the third section cc5ntainfe other comments 

* . • It 



^hich seemed pertinent 



s 



."^ , ^ INTRODUCTION > *' ^ ^ ' . 



w ,The'r^yiew-of ,the "D6sigri,iitg Iristructional porograms" mat^-S^ 

^ ^, ' ^^^•'•f included an an^ysis of , f,ive module packets; a glossary of 

*■ "*!:>':-. - / — ' - V A ■ . 

' terras bo-oklet-^nd>^a document entitled. "Curricul«m Analyzer " ' 
• ^ •* V- TI^Ij^ associated .training materials, 'i.e., "Determining In^ 
\ .structi£>nal Pugposes" and ."Bvaluation for Program Ijnprovei\)enit" 

• ;were not; 4^onsi,dfered ^^h^n the "Designing Ins-tructiqnal Prograips" 

- * J ' * \ . * • ' ^ *^ « * V ^ >^ 

^ . • modules were-rev.ifewed . The reviewer is biased,.' positively,, toward 
' ' • \ < * . . , , , • ^ o , 

the use" of training' materials to assast with the man^' problems, 

faced'^bY educ'ators . . Most educator^ rieed new attitudes, skills / ' 

knowledge. These biases understandably influenced the analysia* A 



V 



MODULE 'l' AND, GLOSSARY 



The Qaals and' objectives of the first module are ^^tated 



clearly, and the game^appears to be a useful tool i^o aej::omplish ^ . 
the desired ends. ^>?hile I have not played the game, hav4 talkpd' * 
wit|i people who have, and ^hey have agreed that the topi. is ade- ' ^\ 
quate^' Cleat" ly, the many options^ cmention^ed in the gan-?, as the 
teams progress up the ladder, shouLa,/be benefi-ial 'zo the users. 
Because of t^he limited choices, however, tfie teams' may not: con»- 
^ ^ / sider .all of the possible options. * • ' ♦ 

The ^0ame illustrates nicely how the complex parts fit into a 

\ . 



i. 
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1 addBc 



^ ^y\ole. .Ev^'jn io 'the mini-ver&ion of program designing, 'the user 

should be al^le' to-^see how the decisions .on one x^ng of the 1^ 
...have a )?e(^ring on, s<ib sequent operations. The. gaTie,s?puld be very ^ 
. halpful in. giving «the participants cbvamonlf'ta-ne of ref|ce'n<:e bo 
g'Qt at che:r di£.f:icult\a3k-. ' • ' _ " ' ^ 

Tosome individuals, the game wj il prob'alDly appear to be ir- 
^ relevant to the task of designirf.^ ins.tructiejnal .programs for \wo 
■ ^rea/sons-: 'jl) Because it" 'is a game, (a part o^ the description , 
^ indicates the simulaJtiqn is less than "real l,ife, " page-17), knd- 

{2)* .because t?he .directions' do iiot communicate what is reailv sup- 
^' ' * ' " . " " . , , . ■ . , 

posed to happen in the game , (at le&st, they' do not communica'te * 

J:his'to me).* f?or tHis latter reason, it appears that a trainej3 

consu-ltant. would need to accompany the package. If the .^ame'is 

expected to stand by itself ,. some add.itin.uil developmental wock ' 

is.\neC(Bssary - ^ '* " v ' . * • J . ' 

^ " <t *J ' i 

/^^^ , ^ • - • 

T^ie only real ^n^rjTrn \l . hav.-/' ibo;i t the gams is that 'which I 

^havfe for all prograh} designing. Essentially, that concern, evcdves' 
arourtd the tendency to gocus pn splutions to p^-x)blems "bf the past; 
'Shat is,^ I am Very concerned ^t^iae we^ntight ,be developing an ex- 
tremely i|Tiportan£ and pov/erful devi'ce which will assist people to 

do well vyhat they should not ^ doin^ anyway. Perhaps the assump-' 

' * • *^ 

tipns module helps •to cut l^cH tHe danger .on this. 

The; "ring" of the ladder which,.! thought was missing from the 

options had 't6.7^o with in*-serv.j'ce» education for those expected to 



/ 



.... - • v.- . . - . .5 

' . • -i 

* ■ . • t ^ ' \ ^ ] • ■ 

implement^ the program; -Perhaps' this wasx takgnX for gfante.d\ ti'be-*- / 

. > . • V ' " • ) • — V » - ^ . , , ^ 

lieVe;, however,' that we h,ave learned enougfi abqut fn^w curricula ■ '. ■■" 

'• % ■ • • '■ *' . ' ■ . ' . - , ' • 

- tc5 know^they pannot be pged without teacher/s "a>a Administrators * ' 
^ .. '■ " . ' ^ . \ ' * ; 

learning new sin Us, attitudes. ah,d kndwl'edge. 

/^s a, design consideration, I woul(^ suggest tfhdt.each partici- 

pant/ih, the gaifie have a set of. the .rules , a's" passipg. a -single Jaook / 

around to all pJfayers would seem to- be to.b t^e-bonsuniXng*, . ' 

Pfankly., I foujid. that keeping 'track ^o^ twenty-eight rules was ' ^ 

more tMn I could handle ixy 'simply re^din§ gather than learning . 

the% rules by playing* the gamd. \ * ' » ' ' 

, Someplace m the fi*rst module booklet\ I t^elieve* it would be 

a good> idea tO; include, some sort ot flow-qhart picture for the 

. reader; The flow-chart would shbw^liow the'modules were to, fit 

together- -Without this' crutch it is' somfewhat difficult to fctttow 

the 3tep-by~step progression through the tqtal package/ ^ 

< • . ■ 5 . ' * -.. • ■ • • ■ ■ • ^ ■ ' 

* • Obviously, the "Glossary" is an^ important part of the §ame. 
Perhdps some^ mention o-f it should b^ made early so the players 
are" able to glahce thj;ough it.. This would enable them to con- 

^ sider definitions of terms at the same time they becdme familiar 
With the 'game board and the game rules. It would be interesting 
\ to observe a person introducjing the ganje to see how one Individual 

written m th4 book-Jet,^. , ^ , 



approach <^;ould match what is 

' ■ . • ■ ■ i.- 

Probably some' additional introductorir work in the ^booklet is 
necessary. ' I* would guess that. on a random sample of, ten j^rincipals, 
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• . * ^ • V • . -- 4 _ 

opl^ tKr^e would be ajjle ,.to introduce the fgame *>as--i 
* . 2^ the tool;^q obtain tfte. pftentia^'sugg.e^ts ; spme, work/-, 
'-^i^i- needi^in 'this ,arjia. • A / ' * ' ' 

-t\ I'.'i ''Understan^abJy,. if^the game is t-o be Effective-. as' a tOQl, 

. participants wih need to^^anders.t.ind 'the definitions of terms 

\, " ' ' '* ' . * ' 

in'Jhe "Glossary." Perhapl a pre-t'Q.st should bfe given' to deter- ^ 

. mine the "entry level" of the" l4arner, .enabling ^im' , to * skip what • 

■' - . ■ ^ . 

he already knows. -(Parenthetically, I 'dO' a wbrkshop called "Inno- 
^ ' . ' . r . * ' - 

vatiqns m Bducat'ion"^ and -alv^ays administer a pre-te^t . . Educators 

. are amazingly un\nov/.ing ; ) "The definitions in the ."Glossary'.' are " 

•those, generally used, (I ^lave inade notes" iri» the booklet where-other 

. terms are also applicable) ' - • * ' ^ ' 

. ' y ^ . • ' 

I would try to reinforce the' definitions in the "Glossary" •/ 

with a s%ide--tape presehtation offering visuals *qf students,, - 

^teachers and so forth to illustrate the , definition/concept^ , (I 

^happen to, haVe' a*^iide-tape ^Jiat gets aT^tJiis idea, i& you would ' 

I • " \ -. " ' , ■ 

care to see itr. ) Undoubtedly y . the. game will \be. only as powerful 

as the concepts understood by,, the players., I co^d bdxwrong 7. 

about t^-.^'-entlry level" of educators, but just tr/ a feample of ' 

^ fifteen to^ see how many know the differences between wet and dry 

carrels,, and this -might gi.ve Some feeli^ig for wh^t little is 

really known. Perhaps' you 'have^^already done the,.",entry level"' 

asses-sment,- and. m this qase, disregard the comments.' \ 
" • , • \ ' / . ' ■ • 

Two difficulties of the^gante become apparent wffen one thinks ' 

• ^ ' . " * "-i' ' '\ ^ • '^- 

"of the rules and at^'the^sarte time\hinks of the . forcecf choice of " 

. " ' ^ \ '. . . 
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. ^ ihe 'terms in yie "Glossary .For exajnple, tihe term "Logical con-. * 

'sistency" Is used frequently, and tlier:<e' is no*pl'ace I can find , • m 
, ' . thd.s^ defined- This, then, /leaves "logical .consistency'* up to the , 
'vote of the players; an.d ignorance,^ even after discussion, could 
^. mean ^that specific -choicels Were mandated.; .that is, the designer ^ € 

. may/ h'ave been accurate in the first place but^ simply gpt oxxtK- 

^ ' \ • " .* • . , ^ - <^ • 

voxed. While, not knowing^ yoyr definiti^'on foir, , "logical consis- 

t^ency, ** it seems to me i^his voting procediire could be logically 4| 

X7 to .1 " • * 

^ .inconsistent. • » « • 

• . i \ 'In addition, because of ^he fprcex3 choieei. (only '6ne ^ccep-r*- 

' table answer), the team of players .must make bnly orie response. 'm 

' \ ■ ' ■ / ■ V " " ^ . 

If one' looks, for example, v at the "Intended Result of Instr^uctiori" 
oh page 3,^ there is a good possibility that at least two ,inten(^ed. 

results will, with logical^'consistency , be chbsen, particularly ^ '0 

' - r ' \ . 

e at a, time when affective skills seem to^be gairning much greater ^. / 

attention \in our schools. I caB*.^nd erst and the. forced-choice * . 

approach, V bo t: it may turn some people off to the point that* the^-, . M 

become disenchanted With the tool. \ . 

^ Finally, there are a number of^ decision-making s,teps celled 

' , * for in the game.^ I would hyt)ofhefeize that while decisions can be 

, /. made. by p*eople in teams of two to'five members suggested' fpr the 

f gaipe, the problem becomes more^acute as s^a£f'member5 , increase : 

; for example* Ma^'ina Hi^h School in^ F^untingt^qn Beach California , 

has 146 staff members. The key issue^shere is not- necessarily one 
' • ' • \ f * ' 



' "of design;, it. is one of problem-Solving , and 'contmunicat.iop^ That 
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is, people cannpt get around to the des'ign considerati^bns .until 
they can do the iiiter personal communication with each.ptlier. Any 
consideration^ ahyplace for helping 'people learn problem-solving 
and communication skills?. • " . , , ' \ 



In summary, I believe the^^toplois wdrthwhfle and potentially 
•* ' " ' . 

• . " * . 

* very* useful . In my opinion; it i§ nots ye,t fully developed if it' 

•is to^ stand by il^self . Please see th^ last? s'lctj/on ofxthis re- 

popt along with comments in the module booklet* for bther thoughts^. 

■ ' ' ' ' . . : ' 

\ , . MODULE II ^ ^ ^. - 



By all means 



Module II should be retained* One ^f the big- 



gest voids in -f* ^J^Sram designing has to do with t^he inconsistencies 

... » > ♦ 

between philosophic a^umptions and 'the actufal activities carried 

out in the classroom. Trr^material used to ^accqmplish the objec- 
tives for Module II is excellent, from my point of view* (Frankly, 
if something in the total program had to be cut, I would be m^Drje 
•inclined to cut the game*) Since each module is designed-^ tp ac- 
complish different objectives, I really do not see. how' either 
could be cut and still maintain the 4ntegritLy «Qf thia program • . 
The second jntodule is especially well done. For example, the ob- . 
jectives are clear, the activities appear to*' help pjj|^rticipants 
achieve the objectives, the strategies for learning -in^ lude 
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indepaaclept stujiy, diseu^.sion, ^nd pjroblem-solvii^g. 

' * ^ * . * / * • . " • . • 

part, of the program* is designed to give, the user specific fe^d- 



ic 



dnally , 



back on whether or not anticipated results have been achieved. 

^ At fi^rst I was somewhat skeptical of the four parab<lfes (and * 

> ^ * * ' * — Nrf * ^ ^ ^ J' / 

theredtl-.li.es a little ptoblem, i.e., getting^ the. user to proceed'* 

past* initial negative reactions, so he, can^ decide whether he doe^' , ^ 

ox does not: accept the .sirtiflatipn exercises- as^ viable 1, sus.pec^ ^ 

the personality of . the Coordinator will mak^tGorisideral2le\^uif fer- 



ence here; if the participants accept, the Coorci'inator* as knp,wledg 



abl^, the. parables will \>is accepted. If the use^s have little re- 

spect for the cbordinatofr they will I'ikely reject the "elementary 

*. • » ' i * , ^ 

-school" parables (win a -few * lose a. fewl)*. ^ • . - 

' I-n my estimation* there are ^twb -voi<^s, in the module: (1) 

techniques to learn about decision-maki<Sig and problem-solving and 

(2). techniques to improve, interpersonal communication on the part 



of the team members. .To omit these or to take them for grante'd is^ 

' . • s *^ . ' : , ^ 

naive. Perhaps t*he "Handbook" for -the , Coordinator illustrates 
strategies for getting at thesq two factors. If not, I would rec- 
ommend same'. (Incidentally, up to this juncture, I really have 
not figured out who- the Coordinato-r i^, what training he/she has 
had, or wheth&r or -not there is %,-^ecific handbook or set. of pro- 
cedures for this^'person to follows I' win focus on , this later, 
but I wanted , to br'ing if up at this:"point because ^^odule J,I fre- 
quently mentions tb^Coardinator. ) » * 

I am espt^cia^lly concerned afcout skills necessary tq get^at 
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a consensus; t>^re aire certain activities to .help people J.earn 



how-to do this, and I believe the package would-be enhanced if 

the Coordinator knew how to do this and built in the option. 

Most of the terms (jargon) u^ed In the module should be 

clear to the participants. ,1 mention this only becayse it is 

another way to' turil people off. * - 

/ * * — ' ««• • • ^ 

I believo the approach 'followed for getting at assumptions 

> ** > ^' . * • 

is good.. Frankly, it seems to nrcTTfiat before we can get^ at 

values for "others," one must have in nrind what values one holds , 

.for oneself- It is because .of this that if I were doing the pro- 

gram, I would have started with ^ some personal values clarification 

rather tha\i the 'concern J|or "man." Only by knowing^ who I am, 'my 

■ ^ / ^ ■ ■• ■ ■ . 

values (real vs. stated), my concerns for changing et cetera am I- 
able to consider the more global issue. Perhaps thisfis a small 
de;tail, but at lec^st It is v/orth co'nsidefing. . . 

I believe the interprejbations at the conclusipn the "Sug-- 
gested Respons^*' section- beginning pn page 57 ar<e very helj^ful.** 
They help -to establish closure on the important objectives m the 
module,, giving the reader comfort in knowihg »heAhe .is^^tuned in 
to. the task befor^ going on to. the next'section . Thxs portion is- 
really quite Weil done. Also I liked th^ added* f lexiKiTlity meri- 
tioned at* th^ Bottom of page 63* This is an improvement ovfer the 
game/.vihere there is a forced choipe of,^optiorfs.» 

The "Prepared Instructional Program Outline*' beginning on ^ 
page 65 is outstanding. ?'here may Tpe a few-options not included*, 



.although they^do not, immediately, come . to mv mindv. ^Certainly, 

4the« devicdi^ids^ the individual (or team) as 'the cfJmplexity of tl^e 
•* . . ^ ' ' • ^ 

^ issues. begins to evolve. , Clearly, the systematic ' approach should 

V, be helpful; unfortunately, there is too often the problem that. 

• ** * * . 

' new designers fail to consider all the potential bugs in the sys- 

I » ■ ' / ^ ^ ' ' ' 

^ tern until they are too far 4nt:o the ball game and the t^sks become 

overwhelming: The, outline hel^s €(i avoid this matter- 



In 'conclusion, I' am very exciteb about 'the "Analyzing Basic 
Assumptions" module.' It is quite w^ll done.; 



: - -MODUL&-rri 

I have' considerable difficulty with ^i'odile III. Frankly, I 
would either reduce it considerably or eliminate- it a*ltQget^er. 

/ ■• - • ■ 

Undoubtedly, some of the' joints made in the ijiiodule are quite 
worthwh:We. With these given, let me suggest thoughts for possi- 
ble changes /to the material. . , ' 

First, objectives should not be written until the entry 
level of -the learner is considered. One^f the problems ^i^j^ J 
Popham's bank of educational objectives at UCLA .(and he is <Jme of 
the first to- admit this) is the problem of people writing in 'to 
use the objectives and, in fact, incorporating them lirra the cur- 
riculum without giving thougH;t; to actual student needs-- I do^^not 

believe the statement m the preface that "participants should ^s- 

* * * 

sume that a 'needs assessment' of educational values has already 

^ . • ^ - ^ ' } ^' ' 

. . ■• f -10- 
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bben conducted" adequately provit3es the, caution sign. 'Some sec- * 
tion needs ±o be "developed on diatg'nosing stuuent' levels, o'f- readi- 
ness. whethef^e stucjents are adplts rSx childiren. ' - - ^ 

New topic: Whil^'JE will mention* this point further atf^ the 
.close of ^ this report, .1 would lik^ to suggest^- here agaan that 'each 
bookl^^t?contain--arcTia^^^ ia ,tKe pages which lUustrates how* ' 

.the particular module fits into the' package. This gives *tfie user'^' 
a sense of from when&e he/she has come and where 'the patV.is lead-^; 
ing. It is difficult^ to get th6 Gest.alt'l even after playdng the 
game, without this road map. 

If individuals- agree '^hat the objectives .for the module ar^ » 
relevant, the. content of the module certainly is written in sueh. 
a way that the user has a* chance for meeting the. obje.ctives- >Ip 
fact, there may even be an overkill in one sectibn- -For example/ 
if the usfer arrives at the answers to the questions on^th^e "Goal - 
Answer Tall^f She^t," page 43, then the user is expected to con- 
tinue, essentially, with the same process 091 the next pages. I am 
not -Certain tlnis is nepe*ssary. It seems to me par.ticipanti may 
learn more about the penguins than^ they really want to knov;. 
Once people have demonstrated competency, there is no strong argu- 
ment to have them continue the same thing ^ ^ 

* f 

In this same realm, I am concerned* that the taxonomy exer- . 

cises may be more academic . than practical. If the u^ers-were in 
a course at some -university; the directing an<3 breakmg'-down-to-- ^ 
the-l.owest-common-denommatpr activitjj^ might be^ worthwhile. 1 
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suspect, >iowever, thai ^curriculum designers are moire concerned 
about getting on with 'the' task .than engaging in Ynental 'exercises . 

. ' . • 

. .pv^n if the aoove position is .i^t accepted philosophically, 
. * - • * • " • . 

I wbuia like ^ to fire one more bit of ammunition which may 'l?e 
. worth 'thinking alDoub: While it seems we might be helpful by 
^J^reaking behaviors down into cognitive and -affective areas, I 
\vould 'ui^Que th^t no ^action. fs totally ciffective .or cognit.ive* ' In- 

deed, we neithef learn wit/hout feeling nor feel without ^earning 

/ . \ J . \ ' — 

see*no placfe in the module where this; point is 4nade*. Under- 



< st:*andab\^y^ I believe you should do this. " . - 

■ . ■ ••. \ ' . . ..." 

Another philosophical concern^I have is me^ntioned m ,page 19# 
\ i*e*, getting parents and other community members' involved. I' . 
* believe it is too''^ late at this ^juncture to have them participate; 

•rather, they should begiii participating during the assumptions . 

modules... 'If the goals and objectives^ are to ehierge from the as- 

sumptions (philosophy) , then parpnts apn children must get into 

" '-. ' • - 

- thp act earlier. ' ' 

^ • ■ ! . - ■ 

I douBt that a real isaue needs to bfe made- about the' sophis- 

' / ^ • <^ \ 

*• tication of goals- Rather than wock so hard -on this^^i^sue, I :be- ^ 

lieve i4 would be more helpful to accept global goal statements*, . 
* * . • \ ^■ 

bi^ then to ask the question, '*What evidence is' acceptaljie that 
the goal/objective has been achieved,?*' This seems to be a^far 
more reasonable issue, as it presumes that teaahers.are aware of 

entry levels of individual childre::i. , T6o often when performance 

*■ 

object j-ves '^ate written, people see the means as an end and fail 

- I * 

\ ^ Si - 
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ta^ really i^dentifi^ ihose evideiiceS that indicate .student growth''/ . 
^•^I am more concei^ned that people demonstrate competency than iJL-r 
lustfa^te they have mastered *thfe mechanics of writifig objective^ • 

Finally, the 'four items on page 49 seem somewhat esoteric; 
for^^x^ple, wha£ does .-logically, developed ^nd internally 
consistent" mean? How does one^ knoW y^hen this ha3 been achieved?"*, 
..The^statenlent "./.each class fhould^be purely descriptive and , not 
Imply a ^system of hierarchy doe^ not really say anything to me. 
PerJi^^s it:, doe's cto- other pe:9$le. ' 

' • .By- the way in spite'-of all my criticism with this module,. 
the chart of jJa^e 56* is quite fcfood*. .1 expect^ that .>some kinds of ' 

tools could be built around the theme of the chart because it 

• ' ^ / ^ 

'shcAvs hbw^ to get at the- "Evidence"' assue..^ 



You.' may 'be interested, in additional comments' in the booklet. 

• ^ ^ -i * , "'^ 

^ MODULE JV , " 

^ . '^^^ devices presented for 'analyzing resources and con- 

str^mts appear likely to be useful to school staff in pursuing 
such an^ analysis/ presuming they want 6r feel ^the, need for such 
an analysis* Frqinkly" havinfcf wqrked with k variety of cjjrriculum 
development groups, I have not found them especially excited about 
ors interested ..in making a dij^ect,. fron^tal analysis of* the poiir- 

tijcal scene; rather, t*h§y have used such devices as the force- 

s ' 



fi^ld analysis, prpbleirt-rsolving procedure? to identify co^nstramt 

[ *^ \ ~ , " K • * 

and ,]^oSsible ways* for overcoming same'. ' , \ 



s 



It seems to me one major void omitted from conp*idera- 
tion has to do with the "teacher training" *eiLement, -JV[a_Jcnow 
enough from history to know that few, if any, pfogjrams can .be 
installed in schools without appropriate teacher training. ''This * 
item is omitted both from 'tUjg. discussion in the preface as well 
;as *the "Preliminary Assessment Foi^s . " At the very least, I 
would add a paragraph indicating this factor has not been included 
in the module even though in "real life" attention must be given j 
to the matter. On page A there is a. list of items which is use- 
ful for doing pre-assessment • This is one of the points where I 
believe the teacher-training factor should be mentioned. ^ 

Even if the procedures as outlined in the exercise were 
followed, I have not been very many places where teachers had the 
prerogative to trade off* the kinds of resources mentioned m the 
exerci4^- Building u^this kind o^ false hope might be devastat- 
ing in tn^real-life situation. At least sOme word of caution 
should be made about this point I 

The objectives on^^^^fjge*^ are well stated, and having par- 
ticipated in the exercises, I believe the content teaches what 
the participant is supposed to learn. Again, I believe a chart 
would be helpful here to show how this section fits into the total 
scheme of| things (page 7) . One good part of the package is the 
help it gives to participants to fearn the common vocabulary. 
This helps them use the same terms as others when discussing cu;c- 
riculum design facets* 
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The exercise included in the a^J^ion on |>ages 29 through 



43 dies not seem^ to be particularly useful. This jud,gment re- * 

fleets my own value system, but I do. not believe .busy educators 

/,have the luxury to explore, the" esdte-ric and minute features of 

each part of the task. Rather than beiflg overly specific, most 

people are concerned about "gett-xn ' on with i, t "' This is simi- 

lar to my bias about the taxonomies discussed earlier, i,,e., in- 

» 

teresting but not especially useful. 

By the w^y, one, ad^i^ional point on the "Preliminary " 
Assessment Form" seems pertinent. Parts 3a and 3b illustrate a 

* point called persons-periods. I have n,ev.er heard of this term/ 

" . ' * 

and I ^did not see it <3efined. Does person-period Prefer to 

» ■* , » 

steadier -period? If this is the case, why not explain it some- 

where? - * 

couple of places in the exercise on "trade-offs" the 
suggestion was implied that money could be borrowed jfrom teacher 

*v ^ . ' «^ ■ 

salaries. doubt if in real life this little matter works out o 
/tlVis neatlyr That is, ^ with the single-salary schedule, the money 
to be spent on the teacher is a function of longevity of the 
teacher and< not a -inatter of cost-benefit or o€her methods to re- 
.deploy existing scarce resources- There is always the argument 
that just understanding the redeployment coi^cept^is worth knowing, 
and that may have been the position the authors wer^ taking when 
the module was developed- I would certainly support this later 
positibn. 

. -15- 
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I have some additional philosophical concerns aboat.vthe 
editorializing beginning on page 122, For example, at the top. of 
the page there is a point made that easy and open schools lack 
structure. In fact, I believe you will find that Vc^ "open" 
schools (not necessarily the free schools) require more subtle 
structure than the more traditional schools-. Further in the next 
paragraph there is a suggestion that commuji'ication helps to avoid 
distortions^ at either end o^ the philosophical^spectrum, I would 
'argue ^ that this suggests a sii^gle rather than pluralistic philoso- 

r * 

phy. For me, xt seems the latter concept should be suppo);:ted if 

^ 

one really needs to editorialize in this section. Finally,, there' 
is a discussion near 'the bottom of the .page about report cards 
tjbat is, perhaps, less than accurate. Many changes have been made 
4n the student-parerrt reporting systems in the last ten, years, and 
I believe the example offered^ is more atypical than typical'. This 
part of the write-up.^akes me wonder whether the author has been 
out in the schools^ recently to know what things are happening or 
whether the qiuthor is writing from, an ivory-tower positions* (I 



realize what I ha\>e said here ijs reasorvably caustic, but I cann&6 
think of a faster way to turn off an audience' than to los^ credit-, 

ability. Tlie writing comes close to dointf' this to.meVand I be- 

' ' , ' ^ - \ ^ ' • ^ ' 

lieve the paclcagcS is potentially too jbelpful to purposely cause 

people to turn off. . * 

\ ' ^ \ ^'^ 

On page 142 tihere is an analysis, made ,of a certain politi- 

/ - ^ . • J/ ■ 

cal situation^^as peotyle perceive it to exist. I would of fer 'the 
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suggesta?s)i3 that -one could -take th'e same basic data and have -them 

•perceived di'f ferently by different people. Ask four different 

peop^le th^ir opinions about the political structu're of a- school '' 

i *■'-./ - ■ 

system, and you invariably find, four different perceptions " li:- 

' . . - : . ' ^, ; • 

respective of what tVie "true" picture might be. Even wjien a 
' ,' ' ' ' ■ ' ' » 

.teacher knows the political' structure, there is little, <if any- 

thing, a single individual can do about the situation. Therefore, 

am not certain what the ekercfises do to enhance the package. 

Clearly,, people ought to understand that political constraints ^ 

exist. This might be handled in a -paragraph. ' % ' , 



f 



In summary, while many parts of the "Assessing Resources^i 
and Constraints" module are liel'pful, I would recommend that the 
total, package be reduced in magnitude aboxft 50 per cent.' You 
may be interested in other commen>ts contained 'in the booklet. 

* ^ - MODULE V and CURRICULUM ANALYZER " 

'feenepally spiaking, I believe Module V and especially 
tjie^ "CurriouluTh Analyzer" will be quite useful to school person- 
nel.' It he'lps to synthesize the earlier elements of the program 
jinto an integrated whole. It is a very nice piece of work; as a 
hiatter of ^fact, the last section with just a little embe'llish- 
|ment could probably serve as the entire packagfet including the. 
one on financial constraints. If I were using the program! I ^m 
certain I would focus most of the attention of the learners on 
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the contents of Module II and Module ' * ^ . 

' Let me g^t more specific with the cri^^ique: The pack- ' 
age is- probably useful for teachers, but it seems Ibo me^it could 
also be helpful for members of the community who v^ant 'tcget into 
the curriculum- selection process bxit* do not have "the skills re- 
. quired. Do you purposely want to ignore t^e "communiff^'';- issue?"; 

Certainly the objectives q,re well_ stated, as usual, and 
the ^'Curriculum Analyzer" ^cai^help a wide array of audiences ad-- 
dress pertinent curriculum question^ .in^ a systematic way. I do 
, have- so^e qualms Vith dne ""of the objectives. The first objective 
on page 3 suggests that the commercially-prepared *descr ipt j^ve bro- 
chures 'are useful for analyzing curriculum offerings. Experience 
.leads m^^tb believe that this is orie of the poorest sources. When 
those descriptions, are prepare.d by the publisher, the propaganda 
is sometimes less than accurate. Perhaps ^some distinction should 
be made alDOut the apprbpfjlate place to locate information which 
pulls "together, ideas 'about Curricula.' (Note should ali^o be made 
that not all curriculum summaries are organized in a fashaon sudh 

• * ^ * • * 

as that produced by the Far West Lab or by certain other agencie^J 
Perhaps the .'first screening could be accomplished without ha^Ving 
hailds -on experience,; subsequent efforts dight be completed with 
actual materials. - • • / ' 

y'^ . I|^believj3^ it.wpuld be helpful, too^ at thi^ point* to tell 
the users that the curriculum analyzer ej tb^r, can or cannot be 

^ used to evaluate curriculu^n materials prepared by other classroom 
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^ te^ehers • Most teachers whc^haver been around awhile and who have 
some raoxy have figured -out a variety 'of ways to put together con- 

. fcributions froiu alternative sources, making a short-of eclectic 
sinorgasborg which turns out pretty well, I can think of only two 
or -three source ^books whicK^give any note at all that more than 
one approach is'^possible. ../and that-^'is whSt-^the publisher is try- 
mg to sell \ \ ^ ^ - 

: , ■ ' \ • 

I was happ/*^ see in this\ section some concern qiven to 
the idea of in-service education. Bit^ier the consideration was 

. * ' • ' ' ' ^ » , ^ 

omitted m other modules or ^t v^^s sq casjuadTly brought to light . 
that it w^s not npticed. I do bel4ev<s it is an important matter 
that needs attention and believe a6me tliought ought to be given to 
the ^issue X\n earlier modules, 'i.e., the financial section/ > w 
r thought the selectibii of the tV70 different types of i f 



^ 1 -cnoi^ _ 



reading ptpgr^ms was q'Site good. It gave people a very different). 

NT . ■ , . » r 

option as <-o what might be selected. (I do not know whether the 

EDL entrj^ has been used at the third-grade, level but 'I doubt it; 

^ • •■ 

and unless they have something quite* nfe'w, there is no way I can 
ijmagine it as b^^ing useful as a third-level reading program- I 
doubt that many other >eQple would think^sb either!) ^ 

\ The one possible diff^^lty with the select-ion .of the 
EDL program has to do,, again, with the credibility of ^he'infor- 
mation put out by the' pub;Lisher". . I realize .the , lab' stale f members 
prepared the document that lisE^ EDL as a reading program, but I 
doubt that they took the time to actually test the program at the 
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third-grade level and simply accepted the comments^rekented by 
the publishei*. The British concept i^ useful not o*nlyr^s an op- 
tion to EDL but also because it gives ^most public school people 
an opportunity to consider the poli't^al climate, the program 
suggests it is legitimate to have children as non-readers as late 
as the junior schools. There are prbbably few places in the 



I . United States at tfte present ^time vthat would accept, this concept, 

/ \v>soyif the two different programs were considered together, all of , 

the baases might poinb*, educationally and philosophically, toWard 

the British concept Tlbut the political climate might negate thi^. 

^ 'In spite o"f the^e comments, I would stick v^ith the programs .inc 

eluded. ' ' ^" • " ' \ 

The optional activities suggested at the conclusiorfNof 

V tlae nnnodule look as if they would be especially useful, particu- 

larly if people were unable to make up their minds about which pro- 
\, - » • 

■-■i : % ■ ■ ~ • ■ ■ . 

grajn to'' accept. I pm, howevej;, not certain how the developer ar- 
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rived at 'the value "7" as the indicator of what to ap_ply to> the 



•different scales. You might want to mention someplace if this is h 
just an arbitrary value. 



' ^' Let me move on to th^^^^urricul^m An^alyzer." ' As far as 

I can tell, ^'the^re are no omissions in the curfi^ylum chairacteVi^r 
tics iTfentioned . V *Many guider, ^such as this frequently omit two ^ 
^ categories, one ^has to do ^ith the amouitt of preparation tjime ^ 

, ^ ' * V "-^^ ■ ^ ' ' 

necessary for a teacher to -use. cxirricuia and the oth'er has to do 
... '.\ ) \ -20- . - / 
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with the s'p^cial skills*, (including, in-service education) needed. 
I was. pleasantly surprised to find, that both of thes'e areas were, 
included in the^^analyzer ; ' ' ' 

- V, ■ . . . 

■As you can -see, although I did not have a "stahdar??" 
^c\|rriculum against . whiph to assess the different module's .that 
ha^e-beferi- put together, in the training package, . 1/ did "rate the 
lab, program. 127 out of a possible s^core- of>. 196 . Actually, I. see 
the 196^ score as being important only if one accepts the value 
of "7" as relevant for each characteristic .and I db not.' Per- 
haps a better (or a*t least a different) way to use the tool is 

* * 

to take a look ^t the difference between the r'ati-ng score and the 

wfeightVthat a person giyes each of the characteristics. If 'one's 

^ ^. 
expectations are not vefy high for a given characteristic, then 

* • 
the difference does not ^become alj! that important.. There were 

tim^.S, for example, when I rated an item highe/ than the , weight - 

\ 

value given later. For those where the weighted value was higher 

than^the rated value, I would have 'cause for concern, and this con- 
♦ - ■\ ' ' ' ■ ■ < ' 

^cern is probably more realistic than the difference between 127 and 
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Please note page xf<in Module V. I really believe this 
is an important^ j^ea, but I saw nowhfere ^in the design of the 
package where this 'was possible. i know' there, is much -to do with 
teachers and* administrators., put if alterhati^ves (different -phil- 
osophies) are goin"^ to be realistip, parents need to learn the 
skills too. Per hap^ when -we take some of the, hocua-pocus put of 
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education; we ^ill have^ greater strength with the general public* 

The "Curriculuni Analyzer," I bel.\eve, has gr^at poten- 

,tial ^for this* By the way, I believe the suggfestions indicated 

that if the characteristic was' rated "0," then it should not be 

multiplied by a weighted value* It seems to me tha't ther*e is al- 

ways a good possibility that something might be rated "0" and be 

very dippqrtaijt to the, potential user* Perhaps the rating/'scale ' 

t ^ * 

should go forofn -1 to +7 or some other such arrangement, to show 



4 



that an item miqht have a high weight but low or zero rating. I 
can even envision that what is available may be worse thap having 
no information! at all, potentially a negative value. The current 

scale does not really permit this. ' ^ 

As can be seen from the assessmeri«t^ there is a good pos-- 
sibility that I would use the inaterials if I'had^ the opportunity, 
*but not because the congruepce between the potential point^^value 
and the value I came up v;ith was particularly goo^. . The i selection 
would be made because of the relative few cases in which the value 
I applied in my own judgment v;as higher thai} what f believed the 
modules had ^to offer. 

There are more comments for review conta^rf^d in the book- 
lets. • * / ^ ^. 

\ - ' ; 

OTHER RESPONSES ' 
The folloi^^i-ng responses are offered in direct ans^r to 



• ' , \ ' • ' . ' ' . 

the questions eirumerat-ed in the letter ^at'ed ^August 29, 1974, 

excluding question 3, v;hich^ I believe, has already been answered.' 

*1. The goals and objectives definitely address an im- 

portant ne^d '*f6r school people ^a5 well* as 'members of the con^mli- 

nity w])p-are more and more being called upon to participate in 

'* . * ' 

educational "decision-making. One of the reasons I believe the 

;/ „ . ' • • " , V * . 

:pacKag^' is'"g6od'^'x^^^^^ artythiri^ ^l^e 

complete ajs the ^materials which have developed. Certainly, 
there are two arguments related ^tp this point: (1) The package • 
may contain 'more about some aspects than people really Want to 
know, and on the other hand, (2) people need to b? willing to xn- 
vest som6 time in learning how to dd^ a job correctly when t^ie fis- 
cal and human stakes are' so high. I would opt for the latter po-^ 
^ij^on, although there are a couple of places in the materials 
where anyone could debate the relevancy, of the content; but this 
same argument can be made on any number of curricular programs. 
What turns out to be one person 'e tr.ivia is often top priority 
for a;iother person; hence, the reason to leave the context fairly 
integrated. . / ^ 

If r had the materials available to me, I believe I could 
use all parts with the participants, except the game, and not 
feel that -the experience would be embarrassing to them or to jne. 
Naturally, if I wore tc5 use the ^materialS', I would attempt to^ de- 
velop somejadditional^ support materials to augment the modules; I 
have" indicated these either in .the preceding paragraphs or in the 

■-23- , . 
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'columns in the modu'lei booklets ... / - 

- ■ - - - : . ■ . ■ 1 

. • 2. *In short, there is no question in my mind that an * * 
individual wij.ling £o^invest at least some small time aad thought . 
(20 hours, I gue>s:pQpuld achieve the objectives as stated in the 

materials. There are some, areas that/, hopefully, will be modi - 

/ ' ' ' / ^ ^ ^ 

..fie^^.,, ..but evea oLf i.t- ,were to stand as is, the grodu'ct could be 

useful in almost- any school in any district, presuming two things: 
^ (1) The leader in the district or sqhool is knowledgable about 
the package and can convince others that the process will get 
them where they ought/ to be going, and (2) the interpersbnal com- 
munication b^twee:i the people involved in positive to the' point 
that decisions can be mad6. What I am trying to say is the .module 
.package i§ essential but not sufficient to -pull of ,the desired end 
results. . . "^^ ' * * ' 

Moreol^er, I •can See no easy way of dealing with this lat- 
ter problem, but it would, seem reasonable for an outride Coordi-- 
nator to be av^)are of the potential difficylty and then to be ablq 
to diagnose and treat the ma\:ter re4.ativ.ely so^on , should, the ^ 
situation arise. ,lf I were a Coord'iuiator , I would start off with 



a few warm-up communication exercises to help individuals cyet 

'/ 

about their business. - - / 

V . /*• 
•3..' Already completed. » / 



4. The only major' modifications I wonid make are two; 

V 

a. - A handbVDok for the Coordinator.- I" had the" 
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impression that there must be o^e- someplabe, but I did net have 
one when reading the other .naterials . I am not certa-m what 
plans, the unit , coordinator "has for training other people to"^ r«.e 
. t;he package. This is We of a dissemination issue, and I will 
not delve further into It at this point except to sa? if jJou want 
to have quality control, it would seem wise to establi^ some 
sort of minimum competency levels for the persons serving as Co- ' 
ordmators. , . ' • , ' 

If you do. not. have fiscal resourc-fes to' determine mini- 
.mal performan.ce leyels of CoordinatprvS, you must use your own ' 

professional judgment-'to ch4ck out- cifiteria. When you' dissemi- ' 

^ ■ f 

nate materials, you may want to indicate someplace that for a 
small fee, you will also offer a consultant from* a list of ap- 
proved, people who can' make certain that those using the materials^ 
reach expected achievement outcomes'. 



5. Rather than eliminating sections, 'it seems to me you 
should find ways to reduce the time investment of personnel by > 
permitting them to move, ahead when expectations have been met.'- 
There -are a few places in the materials where there is ovirkUL 
and this is unnecessary. Some ^ople^witl^ reasonable exporie'nce 
will tune ^out when they believe they are investing their time and 
talent on sdmithing they alreadjr know. (By the^^way/- there are ^ 
lot of people who' think they know things' but really do not and . 
about the only way to convince these, people is via some sort of 
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demonstrable test .and -even ^hen a 'lively debate is often pos- 
sible,) I noticed no such proficien'cy options in the materi^als. 
If I absolutely had 'to prioritize the list*of modules, I 

r « ^ * * ,* 

would first keep the fi|th module and "Curriculum Analyzer." 

- - * * 

Then I would keep Module II, Next would come Module III, then IV, 

laCst would Ce Module I, the game<, Perjnaps 1 would feel morji 
positive abojut the "game i^f I were to participa:te in playing it. 
others, have told me it. is fun. From a. i^les point of view>, it 
might offer just the right gimmick (using the term loosely) to 
help make tl:ie package "gi?'. " ^ ^ ^ \ 

6, I do believe the price .tsg y,ou ^have put on the pack- 
age is reasonable. If a school district is going to make th^ 
Hinds of seri^ous deciaions called for, they should expect to in- 
vest initial money into, people and materials. With .the Federal 

■ i ' ' o ^ 

^funds available through 'di^JEer en t titdes/ m Calif prnra through 



3.3 leg?.sl^tion,^ et cetera, there se^ras to be realization that 
if programs are to work/ 'teachers need t^e in-service to do them 
v;ell. The training .package, it can* be argued, is a staf£ develop- 

ment option, i Even if teachers in a given district were not in- 

' . ' " " ' . / ' ' ' • 

volved m the selection of curricular materiatls (aind they usually 

are), ther'e is plenty of meat in the modules for /all educators, 

if they will. tak^ the time to consider what is available. 

' ' . Colleges and universities* are oft^ slow to change, but 

it would 'seem that any curjriculum and instruction people and/or*^ 

• * * ) 

. ' ' H r 



. . the^ coti^DaitatJts* at t lie California County Offices of Education 

-wd^jftj be logi^ral agents to implement, the program^ Incidenta^y, 



the^tegional education service centers in Texcis and the B.O?C,E,S, 
1^ ' in New York are always looking, for paclcages such as this^ Texas, 
, ifKjeed^ now has a^i^w which r^quixes that teachers hav^^en days 
' per year of S^taff development . ' ^ * . 

■ ■ 

K 7. Jt- definitely would .recommend, use of the materials 

^, ^ and would use thert myself -if thay were available. As I h^v^ in- 
* dicatedi earlier, I have a fiav; different ways to do .the 1' assump- 
tions^' package and believe .the personal values and institutional 
values topis I, use are more powerful than the approach taken in 
Module IT, but this probably" i^ my owh bias. In any case, I cer- 
Plainly agree that the a/nalysis of the philosophy of the school 
^ust be the tirst ,step, and there are many'wjays. to do" this. ' 

Moreover, we' f^ueva) would be willing* to help 'dissemi- ' 
nate the modifies through our modest Learning Center, although ijE 
I weye you, I would consid,er PDK and ACSA ^or potent/al dis^mi- 
,^ \\^xoT\ m^that order. * You may have better contacts than >E-r but 
I v^oul^d start with^ Keith Rolse and Art Thayer, respectively, 

/ The orjly other materials and jpro^edt!ires I wouJA3 use, as 
^n alternatiV^I;^ ^^^^ been put together, 'are these* I^have 
^ dev<eloped 'Myself; uijfortunately, jD^LC^use ..f have- nevjr packaged 
: the \materi^ais , th^y are* tisually p^it into and taken out of a work- 
. ^shop depending upoft ^many fac tors ^ thaft 'are not related to very ' 



much of anything, except me. Some of the ar^'uments for both- - .-^1 
choices are obvi ous « Undoubtedly, th6 program is well done. * . 
ijet it out to the people who can use it. 

PERSONAL CRITERIA • . . 

' \ 

"When I look at different curricula, there are usually ^ 
fo ur umbrella areas corvsidered: Aims, Content, Methods, Bvalua- 
tion* The specifics of these are quite well contained in the 
"Curriculum Analyzer," although they are called "qjurriculum char- 
acteristics . " I saw no major weaknesses in the materials in any 
of the four umbrella areas . What suggestions I would make have 
been conVeyed earlier. 

V7hat I would like to do in summary^ then, is to offer 
some thoughts about the program which came to me as the modules 



were reviewed. ^Hff^se are rather aside comments and are offered 

iust for your 'own use. " * ^ 

1. You ^might want to say a few more words in the intVo- 

duction about the authors, especially those parts wl^ich lend 

cre{2M:ability to the work they have bfeen ^doing in the^unodules^ ^ 

* 

With all due respect intended, I have not heard of any of. the 

* 

people other t,han Otto. ^^Then Glenn Nimnicht did things at the 
Lab, people knew they were going to be great because they had his 
nartie on them. Too few people in the field think highly of labs. 



and I would do anythihg I could to Igt^tJiem know there ar^ good, 
people emp3.oyed there trying to make fenmgs better for kids and * 
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2. If some'thing is not going ta be done with the biblio 

graphy (and because it is not annotated), why not just leave it 

^out? Again, this cuts dov/n the size of the package and makes 

) 

every page count. , " 



3.. In addition tO' tl^e two items I mentioned ecirlier in 
th^ report^- I would also give serious thoug'ht to f^^e possibility 
of including conc^sus-making anWor problem-splving tecfiniques 
in the package. These are not^all that hard to do; if the Coor- 
dinator rece^^s special training, you may want to build the 
techniques into' the handbook he/she^ uses . . 



4. Overall,, I thought the wi;iting was well done. Some- 

one had 4Tnade a careful effort to avoid the overuse of jargon. (I 

r 

took the liberty of marking a couple <^f places where it seemed 
the writing was unclear.)^ 



The materials as*they are designed accomplish the, 

/ 

goals and objectives. I must say, however, thjfe format for the 
program is anything but exciting. I do not know who is working 
o^ x.*his part, but this matter could definitely use some attention 
perhaps a systems chart would be beneficial, or 'even some jazzy 
cartoo^characters . Enough said; I expect something is already 
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6. When and if the program is completed, I would try to 

do it in such a way that you have two major components: (1> t^liose 

items of written materials and audio-visual materials that are 

nonconsumable and (2)* those items'that are' consumable. People 3*n 

« 

the field could continuq^to order those' parts which are* used* I 
believe this can all be done and still keep the packaging costs 
within $50. This is not an unreasonable price to ask for what — 
the peopls are getting. , 

IN RETROSPECT 

I enjoyed the oppoi?tunity of reading and reviewing the 

curriculum design materials. My interest, when critiquing the 

documents, was to be constructively critical. For certain, ^ some 

reviewers might suggest throwing out the 'entire project; my 

biases are much different. VJhile the package completed to date 

is less than perfect, it appears to offer reasonable help to 

people in' the field who desperately need it. ^ , 

.It would be possible to continue further development of 

•the package. This is a matter which the Lab will needv to decide 

(as any^.labi must ) , based upon the availability of scarse resources. 

'> 

Fron my point of view, the Model T is ready and should be gettmq 
out where it can be, Used. 

Possibly there may be some areds of the r;port which are 

^..„ . .. . * : . .. : ' 



unclear; shou^^ thi^ be the case, please drop me a li^e or'call 
^ on the phone. Should it be helpful for me. to y^xplain in person 
any of the poipts I have made in writing, please let me know, as 
arrangements can be made for this. - . ' * 



/{ 



/ 



ERIC 



-31- 
\ 



ED 106 91^ . 

' INSTITOTION 
SPCNS AGENCY 



POB. DATE 
NOTE ^ . 

■EDES PRICE 



* descs'iptoe's 




DOCOHMT RESOME 
95 



, FA, 007 169"^ 

; . ' ' . . . ' 

The Def€rlopmeiiit and Evaluation of Designing ^ 
Inst^ucAional Programs Unit. Far Rest Series in . 
In'structionaL Planning. 

Far West Jiab. for Educat4-onal/Ee:^^arch ^hid ^ 
Development^ San Francisco^ Calif.* 
.Rational Inst, of Education (DHeV) , ff^hingtton, * 
D;'C'. ^ / ^ ( . 

Jan 75 t . . 

^1^27^.; Related dqcuments are EA 007 168-172 



MF73O.76 HC-$6.97; PLUS POSTAGE*; 
♦CurriculAim DevelcTpment; Currixjulua EvaX&ation; 
Educational 
'Educationally 

'Elementary Secondary. Educatiotf; Evaluation' 'Hethods; 
*Instructional Prograa^; *Plaiining; Teaching 
Procedures - V 



I uevej.opraen"c; uumx^iua jiva^ua^ion; 
. Administration; Educcftibnal Developmepti 
Objectives'^; Educational Resources; 



ABSTRACT 



Providing a history, of the development and 



evaluation^ this report discussed- oile set of tnaining materials 
^esi^ned'to increase knowledge and underatdnding of school personnel 
in the area of insttuctional planning and management.. The uni^: goal 
is to assist school staff in making 'defensible choices in fors aBd 
content .of instructional- programs by providing Experience- in the 
process of, and tools^ to assist in an analiysis of # basic learning 
assumptions, the maifching of program qhar&icteristics with 'outcomes, c 
consideration of resources available, and tyhe analysis of existing-, 
programs. The m^erials are presented in' five modules: chalk talk, 
analyzin'df ^basic assumptions, .matching programs^ with goals, assessing 
resoucces and constraints, and selecting curricula. Results of 
external .evaluation are attached^ Field testing and evaluative* 
materials &re appended, (Author/D'H) 
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This report is intended to provide a history of the development and evalua- 

txon. Qfaone ofj,the products produced by the Educational: Management Progxamf. * - 
-V " ^ * » * ^ , V , ; . ^ '^^ - ^ 

Reference is made in the j^odj^ of the repon to more detailed 4*eports or memorar^da 
wliid^ (Jogulpent the deyeldpnent and testing procedures juseS.' • Boir^ of tJie more 
crucial documents relevant to product , effect iveness.'are^ pf^ented in' their i ' 



entirety as attadments to the report. • ^ • . ' 

The^report h^ been prepared for two -kinds 'of "^readers. The first are-:thoS^e . 
associated with the National Institute of Eiication, -ja^d posbjbly some potjsn^ial 
us(Srs of the product, who. need to m^ke judgnents aTrout product quality but don't 
.have the time to. become famifiar with all the lietails of the separate' reports 
on the development and testing. For this group., tlie conclusions drawn from 
the various ey^lu^tidn studies are excerpffed in, the body of t^e report. These 
excerpts may be cgrisidered to represent, the , claims for the pj;6duct thaf^e 
Erogram Staff believe to be^^stified. If reader^^are willing to base their • 
judgnients, about the. product on the authority and professional independence of 



those drawing the conclusions reported then the body of the report? is sufficient. 

The secohd groiq^ of readers ^or whom tlie report is prepare^ .is composed of 
the staff o:^NIE and the -Laboratory , and professional colleagues. who wish to 
know in more detail about the quality o'f the evaluation information collection 

and analysis on which the conclusions are based. The attachments to the report 

7 ^ k . 

are presented to meet the needs of this group. 
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Ricliard W. Watkins 

Program Director* 

Educatiorfal Managem^t Program 



\ .T}\e Developinent and Evaluation, of • 



> ' . • *• Desigainig? 'Instruct ional Programs . 



• "'^ ^signing Ipstructional Progranis is oi^ of six sets of trainifig materials / 
devel^oped tq increase the skill, knowledge and ifriders'tanding of school personnel 
in tlie area of instructional planning, and management.* Instructional planning 
is defined generally as 'that area of educational adnunistratiai that "is concerned 
with providing pducdtional opportunities for ichildreii. ^ Tliis ,area mav be fuither 
defined as .involving th§ establishment of instructional, progranji purposes, the 
Resign and implementatipn^'of ^yrograms, and the evaluation. of instructional" pro- , 
grams. TTie materials in Designing instructional Purposes* is directed to tifie 
second of these three functions. . • ^ 

Intended Users ♦ *' ' . '• 

/These materials- are designed for thfese school personnel who are directly 
concerned wi^h th^ design or selection of $chooJ curricula (inglructionai pro- 
grams.) to be in5)lemented in classrooms. 'Hie specific role titles of such staff 



will vary from district to, district, but most often wilL include building 
principals, (department heads, distVict curriculum specialists, and teachers • 
who are? serving on curriculum comnittees. Depending on the size and organiza- 
tion of a school district or system. Designing ^tructional Programs may also 
be found usefu.l by district office administrators other "than those Avith direct . 
curriculum responsibilities," Board of Education meiibers, parents and students. 

It should be clear that the intended user group cap best be. defined by function 

r * . s> ^ . . ' ' 

rather than by a particular role or, title, and ttjat the groi^ could include. , 

not neces^sarily falUng under the heading of administrators.. , 

. ; •* . ; -Si • ' , 

' Vr.oduct Purposes and Description . ' ■ > ^ ' * 

Hie^ goal of this'iAit is to assist in the preparation ofisdiool staff in 



making defensible choices regarding the.ijom and content o^ iifetructional 

programs • Ufa general objectives o£ the unrt are to provide* experiencqg in^the 
\, ' ^ ' ' » . \ ' 

process of, and tools to^ assist, in (a) an'WaLj^sis of basic cissumpVions about 

Students and learning, (b)'the matching of progi^ charactei'istics with intended 

learning outcoiltes, (c) a consideration of fhe.J^birces avail^le for and the* 

, constraint^ imposed 'on program design, and (d) jthe analysis of^ existing, programs* 

(that mi^t be useful. ./ ^ \ . ^ \ * ^ . ^ 

The materials are pt^esented in five moSules1vhic}i might .each usually require 



three to jthree and a li^lf hours .„ Jie .jnodules^are : ^ * ^ 
I • * — , « • * • 

1. ' ChalK Talk , which is an edacational game played by tljree teams of 

^ three or four people. Each team goes through- the step by steg^ process^ - 
of designings an instructional program^ for a gub j^t area and grade ^ ^ 
level chosen by Jhe teant, De'sign decisions are made at^a^nuniber of 
, , critical choice points and must be defended if daallenged by members 
i jof another team. A glossary is provijdfed so that all participants, will - 

/be able tq use terms- with the same leaning ;^ the glossary is also use- 
fut-dtfTlater modules. There is a scoring system for'tlie gai^V 50 that 
a, team, may "Vin," but the real p^ose ^f the game. is to in\?:x)dufce "the 
participants t6 the total design process, and^to stimulate interest in 
. . the subsequent modules. * / • ' ' . , ~ 

2 . Analyzing Basic Assumptions , in which participants and teams identify 
the assumptions that under ly, their instructional plaiping,, justify 

^ their assunptions to members of other teams, and reach a*team consensus^ 
(On.the assumptions* to be used in planning an instructional* prqgram* ^ 
Four parables representing different, mode Is of instruction 'arjLrjused ' 
to enable participants to clarify their d^m assumptions in six major 
areas.' The purposes of tlie mipdule are to C^) pi:ovide an organizing 
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framevtork tq consider and analy:ze their feelings and asstmplions ^out 

*•''.." 

what and how people .should. leaaii./and*.(b) provide a vehicle through"- - 
\irtiich they can arrive' a| a ^onsensus on assun^tions tha^ wiLl peimit 
^*e/fective group planning of an insti-uctional program. • 
Match ihg Programs With Gb'^s . in ^^^ich the piannijig te'ains analyze ^ 
9rganize the goals and. objectives to be "inet l^y an, instructional^ program, 

'and-make judgments about the- program characteris-tics appropriate* for 

' ■ . ' ^ • , ' U 

meetiTig these^ goals and objectives. In effect, th6 module provide^ • 

.^practice using two kinds of plfennm^ tpols .that can be'*useful in 
application to real instructional planning tasks qn the^ji^bP One' of 
these 1x)0ls enables U5ers to organize goals and vd^ecpives stated with 
different aniounts of specificity into a ^et that provides -a iiseful ^ 
base for instructional planning. The second tbol enables the team to . 
make program' design choices that are consist^t with the program objec- 

- tives . ^ ^ - V 

Assessing Resources *and Constraints- , 'in which, the particip&nts Voijk 
through a set of simulated Instructional planning problems* focused on 
the task of estimating program costs, considering possible cost trade- 
oiffs, and considering staff and commmity resources ^and constraints, 
that could affect the success qr failure of ^^^^jpogram;. This is done ty . 
providinj[ exercises in (a) the use of forms, and cost estimation para- 
meters to obtain approximations of staff , materiaiks and facilities costs 
for a given prog'i^; (b) the rating of , feasibility of cos^t trade-offs 

.^png these; and (c) the jLdentification of non-coij: restraints and 
resources* * * * . , 

Selecting Curricula , in which participants are prepared foY and have 
practipe in the use oJ/an instructional planning tool for analyzing ^ 
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.and conparing different curricula tlmt might be aidopt'^ or adapted to 
ineet^prograiri objectives. The **Currjlculum Malyzer'' in this module can' 
bo, used for assessing , the strengths and we?iknesses o^actual eurricuja 
that schools might consider, and, this module in effecJ:^rovi.de§^gi-oup; 
practice in its lase to conpare-4:\^ existing instruction^' programs in 
Teading, • . ' • * • '/ 

Product Development and .Testing ■ ^ ' > ' > y <| \^ 

General specifications for the unit Designing ^ Jngtructiqual ..gi'ogr.aiiis were 
first set forth in 1970\(Banatliy and Jenks) • TlVese specificaticHis. w.feVe deri^d 
fycm the design of an overall system for increasing th^ instructional management . 
capabilities of school staff. The system 'itself had been designed in response 
to a detailed analysis of existing needs of scliool staff, aiid envisioned the ^ ^ 
develppmgnt of several training and speeific application units., as well as the . 
development of planning and management tools for use in the day to day world o^ 
school staffs, , " • * . 

In May 1971 a much more detailed set' of specifications were prepared' as 
a^ starting point for the liiit (Jenks, 1971).. Six major objecti\'es and related 
behavioral objectives \^re defined, and a detailed analysis of the program 
aesign' process conpleted^^^A first statement of the tasks .to be covered in the 
training unit was outlinecFso that t4ie int:errelation of objectives, gleaming 
environments.,^ curricular information, resources, and constraints could be appre- 
•ciated and used in making program design judgments. , ^- 

As work. progressed on the developm&nt of the 'training unit, staff review, 
and tryout ol drafts of some of the developed material, several conclusions were 
reached that ^ed to rather inportant design change? (Otto, Januaiy 1973).^ Among 
these were: (a) the complexity, of the program design* pxocSss made it necessary 
to present an overview of the entire process before undertaking training in 
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specific steps of the process, (b) a strictly verbal -graphics. approach to this u 
overview presentat;ion Wbuld have been too long to accoinplish \>dthin the reason- 
able tine, limitations set lor the unit, and (c) the original design would have 
.put too' much stress cn con^arison and choice among existing alternatives ratlier 
than on the. design of new progg'anis or tlie creative adaptation of existing pro- 
grams to meet particiilar studdht and school needs. 

/s a ^result of these ponsiderations, the unit objectives were redefined - 
to sfine extent, ^d the decision made to provide" the overview using an educational 
game thatyould also s.timulate interest in the unit, provide an. organiz-ing frame- 
work foV thfe rest of the unit, and set the stage for the cuh'iculum committee] 
approach to instructional planning that seemed most realistic. At the same time, 
^it was decided to undertake the development of a device for actually analyzing 
exis.ting\:urficula in .a wax ^^^^ would be consistent with* the principles under- 
lying the mit, rather than discussing how cyrricula should be analyzed. 

Attention focused on the development and testing of tlaese two modules 
that represented a radical departure from previous Laboratory development 
efforts. • - N . . 

Two feasibility tests of the, game ;yere"isc^ducted, one using a nun±>er of 
Laboratory s.taff members as participants and one using graduate students in 
EducationSl Administration at San Francisco State Uni-versity. The game was ' 

extensively revised after each of tliese tests, and several versions^ produced, 

. •'•**' « 

whidi were tested conparatively, again at San Francisco State Ifiiiversity (Otto, 

April 1973)*. , ^ . . 

"The entire unit aL originally planned,^ consisting of four modules, was 
tested during one wpek o^ a graduate course ^at San Francisco State Uai^rersity 
in the summer of 1973. TWenty- seven students who were practicing school admin- , 

is trators 'or teachers, conpleted the unit in four two -and- a- half -hour sessions. . 

w .1 



The time foi; use \>£ the unit (ten hours^ which was designed fb|. iifteen to 
j^pighteen hours, precluded the collection o£ cognitive op belia\loral data. 'Ihe 
affectiVe data collected may be generally described as neutral to negative, 
although the respondents were positive about many features of tlie unit. Many of 
the negative responses appeared to result from the conditions under which. the 
unit was tested, rather than from c±iaracteristics of the uijit itself* Tlie test 
of the unit did provide infoim^tion of value in planning specific revisions^ to 
the unit. A con^lete report on this* field test is included 'as Attachme.nt 1." 

V 

•As a result of this fj.eld test, the unit was extensively revised, including 
the incorporation of the "Curriculum Analyzer*' as a fiftli module. As these , 
revisions were nearing conpletion, arr^gement^ were maJe for an external 
^valtia'tion consultant to .prepare aif evalt^ition plan and instiximchts for.' final 
field testing of the unit. The revised yersion of tl^e untt was given a prelim- 
inary test in May as insendce training for nine ^lementar^ school principals 
and an assistant stperintendent.from'a sii^gle schooi system. Tliis training 
session was also losed to pretest the evaluation in^tnfinents.. Followang some 
further revision of .the unit, it was tested in five situation^ (two miveristy 
classes, tljree iiiservice training situations) in which 77 administrators partic- 
ipated and corpleted the evaluation forms. The school districts cooperating 
inHhesfe field t^sts are listed on page 11. 
^ The external evaluation consultant collected and analyzed data from all of 

these test situations. His complete report is included as Attachment 2. The 

jf " > • _ » « 

following quotations.. from. the report (Temp) rearranged from the original 

context, pre^sent :the conclusions v/ith respect to cognitive outcomes: 

"Ihe three inpoftant abilities assessed were: 

• the alji^ity to select curriculum comparison categories or' ^ 
importance 

' the ability to specify appropriate classroom procedures to 
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" accomplish a given set of 'behavioral objectives 'v ' 

0 the ability to COTipute rough cost.' analyses of a proposed^ 

progran given a set of district guidelines and cost figures," - , 

\ \ ^ • . 

The conclusions of th.e study with respect to each o£ these areas were: 

.''A practically significant expaasioij of th^ ability of \the participles ^Jtd 
select curriculum comparispi^-categofies of importance was achieved. The grotqp* 
' that 'prior to training would liave been content with comparisons on t\^o J:o five 
dimensions. was ^ after training, indicating that up to 17 comparison dimensions 
were essential to doing an expert job/' (p, 13) • r 

\ . ' . ' - - ^ ^ / 

''A more realistic and \d.der set of classroom procedures and conditions were 
suggested by fhe '.participants after exposure to Ihe materials- and experiences 
of the training unit*. Some sigiiificant improvement in the abilit>' of these 
experienced educator^ to SDecify appropriate classroom procedures was fomd^" 
(p. 16) ^ ^ , ' / ^ ^ • 

"Many more of the participant, understood the logic o£ a cost analysis fom 
and Vi'ere able to conpute rough cost analyses after ti^ainxing than, before. . A 
practically significant increase in the ability of, participants to do snci\ 
analyses of 'a proposed? instructional program was apparent*", (p, 18) 

The objectives defined for the unit as tested also, included recall bf 

certain vocabulary and curriculum specific itemsi The evaluation specialist 

judged these tq be !' louder • level knowledge outcomes" but did collect information., 

relevant to them. He reported the following conclusion: , * 

"The results 'indicate that if th^ unit ^evelQpers do want to accomplish 
knowledge/ recall objectives (seme of which are stated in thq ,materials) , they^ 
will have to adopt a different and perhaps more traditional memorization and 
study for examination type teachipg' strategy. Tliis is not cpmpatible with jthe 
developers more over- riding objectives of achieving application type skills in 
the short training period provided, and is not recoirmended by the evaluator." * 
(p. 19) . ' ' ' 

The final recomirendation made by the evaluation consultant was that : 

"Ihis unit should be made available in a distribution version for use m^ 
/college, and school district settings where teachers and other scliool personnel 
'are in training for curriculum design respohsibilities." 

In addition to this systematic testing of the unit, V^^o other evaluation 

efforts were undertaken. For the first. Dr. T. Bently Edwards, Professor 

Bneritus, School of Educatim, University of California' at Berkeley, agreed to 

observe one of the five field tests referred to above, and to report his judg- 

\ ' .11 



ments about the process, tlie unit> and ^ anything else hp thought appropriate to ^ 
note. Ife^had not seen any p£ the materials prior ^to the claiss, so ^ far as 
is known^had never had any JiscusSions, of the unit with those who had developed 

It.. • . ' ... - . ♦ 

'Dr, Edwards' report is included as Attachment 3. . is Essentially an ^ 
'dbserVeiational narrative which does not lend itself to sinple sumuarization, llie 

follovang quotation from the introductory paragraphs (Edwards) does i)rQi^de <i ^ 

^ . O ' ^• \ /. • \ . ■ ^ ' ' . ^ . 

general 'indication of his overall judgment: , . 

.**A11 of the menfcers, of the class are mature educators, holding. enli:ghtened 
opinions abcfut educational issues . With these students, ^e materials'wre 
hi'jhly 5irccessful. ^TTiat is to say, they genei^ated the' course^ content, stimu- 
/ lated clarifying discussions, and seemed to include sufficient novelty fpr'tlvib 
group' of students... To quote one of tlie men, '1 can now adopt a. fresh- point 
of view in working with my school boated on curriculun revisions. * 

As a. second supplementary evaluation effort. Dr. James OliVero, Director, • ' 

^ ^ . ' " • / • ' 

of the Nueva Day'^chool and Learning Center in Hillsborough, Csilifomia was 

" ' . • ' ' ^ ' ' ' 

asked to review the entire unit as prepared for th(^niainLJ[jU^Xd_:tesJ:_(^^ 

above. Dr. Olive ro was chosen for this i^eview because he is ah experienced 

school adninistrator who has also developecf and published administrator training 

material, and who very frequently conducts. administrator training workshops as 

- - ' ♦ ' ' ^ ' * \ * 

a consultant to school "districts or professiphal organizations . ' . 

' v' " • \ ♦ 

Dr. Olive ro's review contained many suggestions for revision, as aagII ^as . 

\ 

overall judgnents of the unit and each module.. KiS written report is >ncl;^ed ^ 
^..as Attachment 4. IVhen this review was^r^^ed, specific questions were posed 
for the reviewer, and are given in Attachment 4. 'fhe quotations below are from 
the answers to the questions, but are arranged (with omiss«iais indicated) to 

-• ■ • .V - ^ . ' ■ 

pTOvide a brief overall summary of judgments about the unit. 
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' ' 'The goals and objectives, definitely. addr(?Ss" an inq^ortant need 
for school people as well %s members, of tlie commiinity\ivho are^more 
and more being called upon to. participate in educational decision 
making. One of the reasons I ^believe the package is gof)d is that 
I am not familiar with anything else as conplete as the materials - 

• 12 
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, which have* [been] deveuloped. Certai^ily, there are two ^argun^ents 
" ' related' to this point? .Cl)>'nie package may contain more* about 
. , gone aspects tlian pepple want' to knOT, and on the" other hand;. 
, * r ' (i) people need to be willing^ to invest some^time in learning 
^^how to do a job coTTreqf:ly when the fiscal and himian stakes , are 
' so^high... . There .is no question in my mind that an individual 
^ * willing 'to invest at least some smal^ tim and thou^t (20 hours, 
• ' gvessi) coiiLd achieve the objectives as stated in the mater- 
ia iais.;^ If. I ^absolutely had to prioritize the list of modules, 
, I Would first keep.tthe fiflJi module, and "Curri'culum Analyzer." 
Then I would keep Module II; Isfext would coiii) -Module III, then 
/ * " IV^^^and last' woul4 be Module. L, .the game^^^ Perhaps I would feel " 
" ^ . ' more positive about, iSie game^if I .were to ;^articipate in playing 
it.. Others .haVT3 told ms^ it is £m. Fjpm a sales point of view, 
it might offer rjust^ the ^i^t ginmick (using the tenji loosely) 
to, help make the package "go", I do believe the price tag you 
haVe 4o put on the pftadcage is reasonable. If a schoor* district 
-is going, to make' i^e kinds. ^<y£ serious decisions called for, they 
shoulii^ expect to-.irivest initial money into people and materials 
. . . , r definitely would' recomnnend use of the materials end would 
'use ti;iem myself if Ihey wei'e available. - As I l]tave indicated 
earlier, 1 have a. few different^ways to do the ""assumptions" 
package and believe .the personal ^values and institutional values 
tools U use are niore powerfl4^•than the approach talceri in Module 
fl, but this probably is iny.own bias- In any case, I certainly 
^re6 that t^e^alysis of ^the philosophy of the "school must be 
the first step, ajid there** are many ways to do this-.. The only 
other likterials and procedures I would use, as an alternative 
to what has bebn put togeiJier, are those -I have developed nyself; 
]unfor^unat^^ because I have^ nfever packaged the materials , they 
/are usua^y^ut into and taken out .of a workshop ^^pending upon 
maity 'factors that are^ npt related to ;very much of anyGiing', 
except me. Sdtae of the arguments for'^both choices are obvious. 
Undoubtedly, the program JLsVell done., .get it out to the ^people 
who can use it." / . , ^ 



. Conclusion 



'Vj9ien thf^main field test^and reviews of the Designing Instructional Programs 



' IL* were completed, some^ additions^L mino9? revisions were made, tiie Coordinator's 
. Hanualvwas poiT|)leted, and'* the text and materials were prepared for final pro- . 
. ductidn. ^All ot the revisions were based on staff e;q)erience ,in the field tests 
. , ' e^Jror tJie reccmmendaticrfis o£ the external. evaluation consultant, the reviewers, 
and those vAxo had. served as 'coordinators oj participants. Ihey can be quite 
' accurately described as "field-based*' r^^sdons. 

The various kinds evaluatipn information available indicated that the\ 
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unit was clearly sufficiently effective to merit publication and dissemination. 
The decision was made that the Laboratoiy should produce and^sell it on an 
interim basis, pending possible interest to publish from a commercial distribu- 
tor* LabpratoiY funds available for such publications are severly limited, io* 
that the Jfinal version has had to be Vstripg^d dov;rt**'as mucli as possible 
cons is tqit with judged effectivieness. No substantive changes v;ere made to 
reduce costs; quite sijnply, the physical appearance. of th.e unit will not be as 

attractive as would be desirable. 

'' ^ . - ^ ^ 

The unit i$ available for purchase from the Laboratoiy at a cost.af $135.00 

for all fhe' Coordinator's materials ^ and sufficient copies for ten participants. 




colleges and school districts liste^^elow participated in one or more 
ts of the mit Designing Instruction'^! Programs • 

Department o£ Educational Administration, San'Francisco State IMiversity 



Archdiocese of San Francisco School S/stem ^ 
Eureka', California City Schools - 
Livermore Valley, California IMified School District 
Upland, California Elementary School Distrtetj 
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Attachment 1 p 



Pre liminar)^ 'Main Field Test ^ 



liminar^m c 



The preliminary %ain field test results, were included in a, Progress Report in 
the Unit submitted to the National Institute of Education in Septenber 1976.* 
The following pages are reproductions of the parts of the Progress Report that 
dealt explicitly with the field test results, / 
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, 'v.. * 111. PRELIMINARY MAIN FIELD TEST" 

A. Procedures "v , ^ • ' ^ ^ 

During each class .session afCSUSF^ three members of the "training unit 

develdpmjbnt team and an evaluator were. present to coordinate training and 
• - . ■ ^ • \ , • ■ - 

collect transactional data,. One development team meipber acted as 'cliiefco- 

ordinator^for the eVitire unit. Since there v/ere but ten hours over a period*. 

of four blasts meetings, to aduiinister ten hours of training, the pre-posftest 

(subsequently cancelled), a^nd affective questionnaires, periodic corrective 

jaction was required to compensate for .time constraints and unanticipated 

events. ^ As v/as stated abovfe, some resequencing of materials had been, done to 

ease the time, constraint's, prevalent a't this site. Such mechanical adjustments 

however, ditf^not allov/ cor\trol of other site characteristics which prevented * 

the collection of cognitive' perYormance data and profoundly influenced the 

affectivQ data collected at this s^'te, v . - , • - 

B. Training Characteristics • . . 

^ ' • ' \ , • ' \ 

During, the tr'aining sessions at this, site each participant played the 
role of a member of arv instructional planning team, performing individually 
and in groups $o: (1) design an instructional .program !for a chosen subject, 
area and grade l^ef, (2) analyze basic assumptions about learning for con- 
sistency with, chpsen instructional methods and techniques, (3) match yarious 



types. of goal statements for appropriateness vtith a variety of inst>>uctionQl 
vlnethods and techniques, a.nd (4) judge for feasibility an instructional program . 
given examples of three types of resources and constraints. In addition, 
participants were .asked to complete a number of take-home exercises that required 
them to: (a) design a preferred ^instructional program consistent with their 
own individual basic assumptdotrs about learning, (b) analyze for possible 
adoption two ^given curricu.U with the*aid of w ctjrriculum 'analysis tool, and 

(c) assess for feasibility an instructional program given information on a 

- : f . ' \ ' / . 

' cur|icjLiluin; and. tem^;5?ctl and financial^ resources and^ cons.traints that might \ 

,bcnef.*t orj'mpair the impl§nefi^|-ttoir^-4J)£L_,px^^ 



C. Site Characteristics h « , •< *" 



, The/alternatie tes,t sitfr al; Califdrri-t^i Stat^ Jnivers.ity.,, S^rf Francisco, . 
consisted of a class in Curriculum aifd Instruction, conjposed of 27 graduate 
students ^^/orking inservice and efrolled' tn an M.A. program, in Edu'cat-iona^ 
Administration. ^I^ians caTied for the instructor in charge of the class Jo 
provide. Progralh staff with four sessions of two hours 'and thirty minute^ each, 



over a period of two weeks, to administgc training and .testing. Since this 
was a^' alternate site that was planned in the event tfrat the' primary site at 
Holy Mames4vas not used, the. ten hours* allotted for field testing was antici- 
pated' as beinc/inSuf/icient. , However, because of the diffiejjlty ^"jiVolved 
in obtaining a more favor^le field test site duril^g the summer months, 
■because of the need of staff to ^/ther as much data on ih& tra-ining unit as 
possible,, plus the requirement of meeting Progfam wprk unit milestones, it 
was decided to proceed with testing at CSUSF. * * 

■Nearly, two- thirds of the field test sample consisted of teachers, at the ' 
secondary school level. Also represenjed were Principals, a Sdhool ^sycholo- ^ 
gist,?. Di-strict Offi ce personnel , a ^Director of a Teacher Center and^a.Curri- c^' 
culum^ Specialist! Such an 0QCupat^n,a,lly diverse group^ was seen by Program 



,taff as a rerescntatlve sa.ple of eventual u«.s of the%a,„ing uMt. ^ 

Three of .the ^art+clpants had had previous e.per1e,>ce.,nth Far Mm Ub- ■ 
iratory-ahd Educational .Management Program products «l,icl,. to these few. _ 
„,ade ^^.^r.tr^ the ajipearance of Progra. staff "and the training unit. _ 
Moreover, all ^^.the lest subjects were acquainted wtth each other which .ade . 
■easier the performance of team training activities. • 

. Negative .ii characterlstlcs/stied la.gel/ .fro» -thflac. of t1« allied 
^adllhlsrer all portions of the tr^^lnlng unlt.'lt. pre-pos«est. and.affectlve 
.--,p^,,4ik_Panic1^^ ^ 

' beginning the^lrk^^^^^^t^^^ ' ' " 

. be gatho;ed .on Module One. Regular clas^ alignments' given participants to 
. complete at hp.e by the f.p.tructor in addition to -taKe-,ho.e exercH» ^ 

panyihg the t^aining unit, pr^venfed proper t1.e and. attention ^eing^glven the. 
"trainlng^rterlak assigned af ho«wor.. In addition, on the.flnal day of • - 
\leld test.lng.. *1cb was' the final day of the course In curriculum an^ Instruc- 

'tlon: the c'ourse instructor resumed con.trol of the Cass after llV^ hours . 
■ to discuss worK accomplished in the^coufse. This Prevented thOo.ttest fro™ 



being administered " ^ 



in genera.1, particlpaots at the test sUe, Seemed Jlsenchanted with the... 
t.s.s thdt the^weVe perfo'^ng. .■.Thef^as.dissatisfaction heard ab.ut the 
.■ p;ogr.:i.w4ch th.y were enroNud. Uboratory staff and the tna,n,ng 
., .it „ere.sa1d .by\o»e part1cipants,to have been used for displacng anger a.d 
T frustration. DestlL these neg^iv^racter1stics._ however. Program staff . 

.ecelW.a total of\ine unsbl.1c>ed-r>uests for the .training unLt for use ,n 
, i„3erv1c: settings. Weov.r; two participantsWeHed that staff consider 
• theli school sites 4 futurete^^tlng of the trai^ng unit. ^ 

St'- < , . 'v , 



D. Data Suniniary and Interpretation . ^ * 

Data analyses were simple since no cognitive or performancej data were 

' * 4 *■ 

collected. For each item on the affective questionnaires statistics v/ere 

computed on a seven-point scale for mean, standard deviatipn, and standard 

erfor of the mean, - 

j^.The standartfv^f acceptability w,as set at two standard Errors of the mean 
obove "4," the midpoint df^the scale of favorability. Thus* it v/as required' 
that each item be statistically significant beyond the laidpoint of th'e scale, 

*If the item mean exceeded this standard, then the results were considered ac- 
ceptabl-e. If not, the result was unacceptable for that to which the item 
pertained,^ ' ^ ^ ' 

Eti^ause of the adverse characteristics of the CSUSF test site" much of the 
'data gathered on each portion of the training ^nit v/as negative, Interpre- 
tations of these data ariftherefore made in light of the site characteristics 
that CQi^ld most likely .have caused negative responses^ ' Where negative ke- 
sponses were directly addressed to content, method, and execution of the train- 
ing,* suggestions for^ revising and improving these are made, ^ > 
/^Two hours were allowed- for completing Module One of the training, a game [ 
in which teams of .players perform tasfe in* instructional program\,design. 8e- 
cause the pretest,' which, requires participants to worfuentirely alone, was ad- 

.ministered Just prion to this activity the transition between the two entirely 
different tasks was judged by; participants difficult to make. In addition, 
the limited amount of time^tvailable for compl:eting the pretest caused frustration 

^Copies of iiffective questionnaires showing itenr meafisj^and Whether each 
^item pa's^ed its staindard of accept*at)ility„ are found in the App'endix, pp. A-7 
to A-33. . ' . ' • . .. 

^Affective results for each moduli are di§p1ayed4 in the Appendix, pp'. A-34 
]to'A-36t, 



and ill-feeling among somo parlici pants; although the game was played wJt^. 
alacrity, this ill-feeling was manifested in affective data gatlie&ed.^ 'As 
a result, only five of twelve items for Module One met the standard of affec- 

tive acceptablilty. * _ ' ^ 

r — 
On Module Two, a writ,ten training activity thcit requires paYticipants to 

'> ^ • • 

analyze the basic assumptions about learning made by themselves and others, 

twelve of fifteen affective items met the standard of acceptability.. Negative 
written' comments dealing with cjarity of feedback responses were mentioned on 
some questionnaires. However, because of the relatively abstract content of 
the module, 'and the fact that it contains muqh simulated material which is 
-open to individual interpretation, it is possible for 'confusion to result on 
the part of some/ Future revisions of this module will be directed at clar- 
ifying feedback, and lessening the possibility for confused interpretation by 
providing diagnostic eXercis^es for participants to 'use to perform some of 
the analysis exercises'with their own input. information. 

- On-Module Three, a written ^activity requiring individual and team decision 

' ' ' X 

making in matching instructional programs with goals, nine "of thirteen affec- 
tive items achieved the standard of acceptabil j-ty. Responses addressed ^to 
niethbdology and execution of the training material A^ere all. favorable. Those ' 
items that failed tc ^iieet the standard referred to clarity ©f feedback'and 
.th'e^amount of writing required, -Ai'-ich was judged ex.cessive..^ Much of tlie 
training in this module deals v/ith'clarifying the rationale used for matching 



%ee Appendix, p. A-37i for written and ver^'al comments on<i this module,' 
^.See Appendix, p. A-68, for comments of ' participants . ^ * , 
'^See comments in Appendix, p. A-39 \ i 



inst'ructional programs appropriately with <.,fterent,typSs of .instruction,! ^ 
goals. Therefofe ™ore writing is required than in ,cco.,panying modules.. 
TO lessen the ambUnt of writing, as suggested by participant responses., • 
Checklists «ill be provided to accomplish the san« purpose as the writino 
exerci;es.. FuHher revisions ,iil'l .also, be directed at clarifying feedbad: 

- 

responses. 

on Module Four, a wrUteh actVvity requiring participants to assess re- , 
sources and constraint that can affect the feasibility of instructional o,ro- . 

■ grams', only si. 'of 'J^xteen affective items met .he standard' for acceptability.- 
Because of time constraints; thi; training modul| .....administered ,'n tr,n- 
catW form with.only tte latter half .c^pleted by pa«icfpants in class, ■ , 

■ Its firu'half. assigned as taewor^.-was assigned-" the same time that, 
participants were'Vequired.by the regular.- course instructor to complete a take- 
home final .examination for their course grade. As a result, few participants . 
completed the, first portion 1, which many a^s/ssment ex.r«ses were necessary 
for adequately.compl^.ing the second part. The fact that its conceptual • 
continuity was. disintegrated partiafe Accounts for m,ch of the.negat ve at, , 

, on this module.' Nevertheless,' other co»e«ts referring to lack of clan tj- . 
of the sch^e Of analysis for assessing, resources and constraints., unclear . 

■ . ieedb-ack. as well as-'the unrefined lethod used in some of the assessment . 

■ ■ -exercises, indicate that M^Sule Fouf must -|>e...revis^d substantially. \ . 
me curriculum -analysis tool, again becai^^' of time constraint,.- was a,|- 

SJ^Mstered as ho„,work-with an .=c»panyi.g .xerci.e that asked participants 
/ .0 rate t» sample curricula with the aid- of the tool. Affective and trans- 
• .ctiona.V data .ath^red on this.portion of the training. unit indicate that most 

' J 

' 9see comments in Appendix,- p. A-40. • _ * 



of tho, negative comments v/ere directed tovvafd the exercise, which wa^ judged, 
insufficient, rather than* the tool. It is. assumed that since the affective 
puestionriaire was appended to the accompanying exerciser-judged confusing and* . 
•unci ear-* there was some transfer of negative feeling from the exercise itself / 
to the ''affective responses for the analysis tool g 

- m * 

* * • 

Negative affective* responses on the tool were contradicted in many ccses 
by verblil participant endorsements directed at its utiljty and potential 
.marlcetability* Moreover, pr^evious positive transactional Jiata gatnei'ed on the 
tool, and the" fact .thajt it was used by several participants dining training 
sessions to refresh their memories on the conceptual fran.ework used in the unit, 
.counter the. fact that only one of fouKeen affective items achieved the standard 
imposed. The exercise, therefore, will* be slibstantially revised, and revisions 
of the tool will wait Until further field test data are gathered. 

An overalT expression of participant interest, and judgment of the 

training unit's utility, was gathered by ^ asking two qu5>tions on the bio- 

graphical information. form completed by each participant in the field test. 

In answe^ to the question: "Qo you, now have in mind<|n_area of utilization for 

any new skjlls you rtjay acquire frojn thjs (these) training units?", there were * 

15 .*'yes" responses and 7 "no" irespor\ses (one participant failed to respond to 

this, .question). In gmswer^to the second question: "Are you interested 'in 

/ ^ 

participating in future field testing of training Lnits?%^there were 16 "yes" 
responses and 4 "no" responses,, (5 failed to respond sind twoyere not sure). 

The above responses indicate that the strong negative feeling manifested 
publically was somewhat reduced' when participants responded privately. They 
also indicate that those having strong negative feelings about their «.A^ 



— 



* ^OSee jcomments in Appendix, p.. A-41,* 
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progr™ in aeneral--a„d the training ^it i„ particlar-^might haJs been 
."»re likely to express those feelings to the training coordinator, not the 
instructor in charge, than were their „,ore affable pe.ers. This' interpreta- • 
t.on of participant behavior is consistent with the fact t^at th"e Program's 
need to collect product evaluation data freqMently runs counter to the pre-' 
dispositions of field test participants. Regardless of these contingencies, 
however, corrective action is required on this training unit before once agaii, 
field testing the product. Plans for such action follow. 



/DESIGNING INSTRUCTIONAL. PROGRAMS ■ . ■ . 
Module One - "Blackboard. Blues" 
^ ^ • ' Affective Questionnaire • ^ 

The developers of this training activity would like to know whetljer you regard 
. the^knowledge and processes covered useful to you .and applicable to your work. . 

It js also importaiit toJ<n6wyour reactions to specific elements of the 
' * "Blackboard Blues" Game. Please indicate your feelings and. attUudes by'cir- 
' .cling the appropriate number to the right of each item- ^ 

Mean 



I • V^lue ofTraininq, Elements. 

1. Are the directions to the. 
Game clear .enough to allow 
you to perform its .activi- 
ties? . " . 



Standard 

^PJAT; ' • • Achieved? 

' MODERATELY COMPLETELY 



2 . 3 



.W 



Yes 



2.. Is the Glossary clear 
enougfif-to allow easy-" 
understanding of the 
definitions? 



.3 • 2 \) 3 



W 

5 



Yes 



3. Does the organization and 

• content of the Game board ' 
ptrovide a compreh'enaive 
format for outlining in- 
structional program design 
decis^^ons? 

4. Do the Handicap Cards 
provide enough of a handi- 
cap to even the* chances of 
competing team^? ' 



1- 



V 



Note: 



In the origina^ versions, sp^apes .'fqr comments ,v/as provided 
for" every itemi The^ symbol ^^show,s the location of the 
mean responsl. ^ The criterion for acjiievement of the^ standaW 
is described ir\' thp text. . ' 



A. 



Yes. 



•Yes 



r 



ERIC 



Methodology and Execution , 

irl*^ As the Game progressed, did: 
* you» find that you' developed 
. ' ^ an inteHst in^-it? ^ • 



2. Did you find the challenge 
• process used in the ffame to 
be interesting? 



NO MODERATE "INTENSE Standard 
INTEREST INTEREST . INTEl^EST Achieved? 



4 5 6 

4 5 , 6 



.No 



Yes 



3. 014 >Bu find that the Handi- 
cap Cards were interesting? 



■4 



5 .-6 



Yes 



4. Did you find that working-as 
a team gave you ^'realdstic" 

¥ practice in' the tasks 'required 
•for designing instructional 
programs? 



4.5 16 



No 



ry. "III.. • Value of the Module , ^ . / 

• ^ * . 1^ Do you feel vthat the Game 
"helped you gaia an under- 
^ J ^ standing 6'f the »prpcess of 
. ' designing instructional 

5 . prqgrams^? - ^ 

. ^ \. D^'id you* like' this modiijle? 



NOT AT 



Mean 



ALL MODERATELY COMPLETELY ♦ 



Standard 
/Achieved? 



1 



3 

■3 



4 5 
4 5 



6 

'6 



7 
7 



No 
No- 



. 3. Woifld you .recommend the 

(aame'-'ta-^thers^inyolved iii^ ^ 
designing instructional 
^ pVogr^ms. •■ - ^, ^ 

\ 'f v 
4. If you ■werte,aB1e to have the 
Game availaBle in, your school, 
c^uid you^ use it^o design 
inltrudtional program!^ for any. 
subject at any grade level? 



i 

r 



,1 



4 5 



w 

4 \ 



No 



Yes 



Sf B'lease" make.'any additional comments" that you have on the Blackboard Blues 
Game'' bel ov^.' , ■ 

. ^ . • • - '10 ' ^ • ■ - . 
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' , • DESIGNING "INSTRUCnONAL PROGRAMS 

f 

Module Two - "Analyzing Basic Assumptions" 
\ , Affective Questionnaire 

The deve>op6rs of this training module would'like to kn^/v&her you 

regard the knowledge and propesses covered useful to you and^^ifppli cable to your 

work. It is also important to. know your reactions to specific elements of 

"Analyzing Basic Assumptions." Please indicate your feelings%nd attitudes by 

circlmg-the-appr^opriate ntimber to the right of ea'ch^tem. 
' » • • . 

Mean 

NOT AT 

■ ■ ALL MODERATELY COMPLETELY 



I« Value of Training Elements. 

1. Is the "Plight of th'e Naciremansi' 
• J an' adequate way of introducing 

*the process of analyzing basic 

assumptions? 



W 

5 6 



Standard 
Achieved? 



Yes 



Is the "Parable of' the Fish" an 
adequate '^/ay of introducing one set 
of attitudes, feelings, or beliefs 
about man and learning? 



3 4' 5^-6 



Yes 



3. Are the sugge^ed responses 

(Assumptions ^d Interpretations) 
for the "Parable of the Fish" 
? clear and comprehensive enough 
for you to understand one set of 
attitudes, f eel i'ngs or beliefs, 
about man and learning? 



3- 4 



5 6 



Yes 



4. Is the team activity h'elpful 
. in analyzing basic assumptions 
drawn from the "Parable of -the 
Chicken?"- . 



5. Are the suggested responses 

(Assumptions and. Interpretations^ 
for the "Parable of the Chicken"' 
clear and. comprehensive enough for 
you to understand one set of 'atti- 
tudes, feelings, or beli'^^s about 
man and learning? 

' ' . . "11 



1- 2 



1 2 



W 

a • 4 5 .6 



3 4^56 



Yes 



7 No 



28 



6. Is. the team activity helpful 
in analyzing basi.c assumptions 
drawn from the "Parable of the- 
Ape?" 



Mean Standard 
NOT AT Achieved? 
ALL MODERATELY COMPLETELY 



.4 



We 



Yes 



7. 



8." 



Are the suggested responses 
(Statements and .Interpretations) 
for the "Parable of the Ape" 
. clear and comprehensive enough 
fdr you- to understand how basic 
assumptions directly influence 
instructional program design? 

Is the team activity hel pf ul 
in analyzing basic a,ssumptions 
drawn from the "Parable of me 
Rare Bird?" - " • 



9. 



Are the suggested responses 
(Statements, Interpi'etations, 
and Revisions) for the "Parable 
of the Rare ^Bird" clear arid cranpre- 
hensive enough for you to under- 
stand how basic assumptions 
d4rectly. influence instructional 
program desjgn? 



1- ? 



1 2 



w 

4 5 6 7 



W 

4 5 6 7 



) : 



4 5 6 ' 7 



Yes 



Yes 



Yes 



11.^ Methodology and Execution ." 

1. Is t"he Naci reman theme an en- 
joyable way to practice the ' 
process of analyzing basic 
, assumptions?' 



4 5^ 6 7 



Yes 



2. Is the writing quality of 
"Analyzing Basic Assumptions" 
adequate? 



1 2 3 4 5- 6 7 



Yes 



3. Do the elements of the training 
■ ^tnodule fit well together? 



1 2 3 - ■ 4 ■ .'5 6-7 



Yes 




. 12 

23 



) 



Value of the^ Module, 



Mean * Standard 

NOT AT . _ ^ ^ Achieved? 

'ALL^' MODERATELY COMPLETELY 



1 * Do you feel that Module Two helped « * ' • 

you g^in an understanding of the * " ' 

process of analyzing basic ' ^ 

assumptions? * 1 2 3 4 5 5 7\ No 

Do you like this module? -1.2 3 4 5 6 )? Yes 



3* If you were involved in designing, 
instructional programs in you/ 
school,, and were able to have ^ '^'^ 

this module, could you use it as ^ 
. a quide to analyze basic assump- * s 

tions? . 1 2 3 4 5 6 7 No; 



4* Please make any additional comnents 
that you hav.e on "Analyzing Basic 
Assumptions" below. 



?'. 



30 



\ DESIGNING INSTRUCTIONAL PROGRAMS ' .' ' 

Module Three - "Matching Programs With Goal^s" 
* / ' Affective Questionnaire . 

The developers of this training module would like to know wh'ether^you re- 
gard the knowledge and processes co\fere(f useful to yoy^ruU^li cable to your 
work- rt is also important to know your reactions to specific\lements of the 
module. Please indicate your feelings and attitudes-by circling the appropri- 
ate number to the right of each item. 



1 ^ 

Mean 



NOT. AT 

ALL MODEPj\TELY COMPLETELY 



Standard 
Aohi&ved? 



Value of the Training Elements . 

. — 

A. Did you find theMciur goal 
levels help,ful -io* the pro- 
cess of matching programs ^ 
Mtth goals? 




D. 



s the "Plight of the 
Naci remans Revisited" an ade- 
quate way of introducing the 
p.rocess of matching programs 
with go^li?r" • 

' I' ■ ■ 
Is the -team activity helpful 
in' matching' Pre-Stated .goals 
for consistency with the pro- 
Dosal and for appropriateness 
with a [Particular program? 



Does' your Preferred Instruc- 
tiorial Program serve as an 
adequate prototype for match- 
ing -with the Pre-Stated ' 
Curricular Objectives? 



<E, 



( 



Does the set of instructional 
objectives provide enough guidance 
for you to select the most appro-^ 
priate options for a reading 
program? 



1 2 .3 4 5* 6 ' ^7' Ye? 



2 3 -4 5 6 




2 



2 • 3 



4 -5 -6 



4 ""S I 



Jfes 



7 Yes 



7 Yes 



7 Yes 



14 



31, 



/ 



NOT AT 

'ALL MODERATELY COMPLETELY 



-^F. Are the simulated Case Study 

and Information Bits on danus , , . ' 
Junior High adequate to' estab- 
1-ish the context, for^ theJake- 
. Home exercise on resour^/ ' 
. constraints? o' " 12 3 4 5 

/ju • ' 

' Does the set of 'Curricula^vbb- . 

jectives adequately dgfine the , V . 

Environmental Studi es Program « / > 

so that you can assess the re- ^ 

sources and 'Constraints gpvern7 * " • . ^ 

ing it well enough for you toV^ } 

judge its feasibility? . '1^ 2 3,4 /5 




H, Are the Suggested Responses 
• for each 'activity^ or exercise ' .Mean 

clear and comprehensive enoggh' 
^ to enable you to gdmpjete the * * " ' . o ' ^ * 

activity or exercise adequately? ^ ^ 

' If not-, explain why below. 1 2; 3/ ^ 4 -5 



\\\ Metli^oloqy an^ Executipn ; " . 

hA Is'-the plight of the Nacireinans ' ^ / 

sufficiently removed to enable , * v ^ 

you to assess -tK^ir prob,lem " . * . ^ . ^ , 

• objectively? 'v/ - l 2 3^' ,4. 5 6. 



7 ' ■ 

B. Given the context of the simulation 



'does it enable you to participate ' . V 
more fully ia tbe tearji activity? 1 2 3 4 5 



**C- Do the simulated. esse Stydy. 

'and the Inforjnatioa Sits estpblj^h 
'the context well enough for you, .to 
assess the resources; and constraints 
^ governing the Envirohmenta.1 Studies 
Program? ' 



*Items F and G referred to the take-home, exercise concluding floduJe Three. 
Data for these items, are found in the section on Module 4, items lYA and IVB. 

^*This ,item referred to the* take-home exercise concluding Module-Three*, i 
Data for this /item ^^re^ound in the section on Module 4, item IVC. ' 

15 . •* . 

' \ ' . • ...» . ■• ■ 



D. Do you think that the- activities 
and exercises demand too much 
wrfting? 



|^ ' V Meccn' Standard 

NOT AT , • ' Adhieved? 

. ALL- MODERATELY COMPLETELY. . •• 



4*5 



No 



E. Is the written style o^"Matching 
"Programs With Goals" adequate? 

• . / 

. III. Value of the Module. 



1 • 



4 5 



Yes- 



A. . Do. you feel that Module Three 

• helped you understand the pro- 
cess of matching programs with 
goals? 

B. Do you. like this module? . 



> 2 



3 
3 



■ W 

4 5 

w 

V 5 



6 
6 



7 Yes 
7 ' No 



C, If you v^ere engaged in designing 
ins.truc©»onal. programs for your 
school, could you use thismodule 
as a guide for matching- programs 
with goals? 



4 5 



No 



D. Please make any additional comments 
,.on "Matching Programs With Goals" 
\below: 



ERIC 
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DESIGNING -LNSTRUCTIOI^AL PROqjV^S 



V 



Module Four "Assessing 'R^oufces and- .Constraints*' ' \ 
Affective Questionnaire 



The developers* of this" training module would like to know whether you regard 
V the knowledge ang processes covered useful to you and applicable to your work. 
It. is ^^so M^t^r\t to know i^our reactions to.specm'^ elements of ^hlp^dule- 



Pleasgnidicate your feelings^and attitudes •by circling the appropriate number ^ \, 
.to the right of each .item, * ^ . , • 



l\ Value of the Training Elements , 

^ A, Does the scheme of analysis** 
• ^ (goal?, decision-making, 
leadership, motivation, 
. monitoring and maintenance^" 
helpryou gain insight into, 
prevailing resources and 
constraints in a school 
settina? 



4 



Mean • ■ Stancldipd 

NOT AT . c v.. Achieved? 

ALL KQDEHATELY COMPLETELY* 



-J. 



4 5 



a. 



>No 



B, Does the "Sumjnary Report" from 
the Principal of Arcadia High 
School adequately describe ^he 
institutional and sociral 
resources that contributed to 
the success of ^Arcadia's 
program? 




V 

5 



Yes 



C. Is the team activity hejpful 
in identifying the prevailing 
resources and constraints in 
Arcadia High School? ^ . 



4 5 



6 ? 7 



Yes 



D. Poes the module's training 
development adequately* 
prepare you-for thexulmin- 
ating individual activity-? 



1 .2 3 



4^5 



7- . . ^ 

6 7 Yes 
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1 



C Do 



^ ' 1 - 

\ /••-.■ 

t. Do*Inonglove's nremo and the 
* ' • ^n^basket" infonrfation' . - 

^ • ^ estab'lish the context ade-^ 
' quately for yoU to^^perform ^ 
^ the ConrespondiHg^'exercises? 

F,. Are the Sifggested Responses, 
'for each pxencise c-3fear^ and 
/ coffipreliensiye- snouqh to enable 
you to resolve any difJ'iculty , 

, you may have encountered \x\ 
"a^ exercise?' • • # - 

; • - : <r \ \ . • 

•'Hi Methodology and Executiof^ 

A. ^re Arcadia High School and * 
■t ' .'Pt)ly_technic High School 

contrasted, markedly enough 
. to 'Provi de you a good per- 
^ specti've. t;o 'view the {potential 
schizoid behavior of resources 
.and. constraints-? ■ "''-^^ 



'y^ Mean 
ALL«--- MODERATELY COMPLETELY 



Standard 
Achieved? 



1, 



2^ 3, 



C 



V 

4r 5 



4 ^ 5 



6' 



Yes 



5 6 



No 



Yes 



Given the context of the 
simulation^ does it enable 
Vou to participate more fully' 
^ in Xhe teim activity? 



4^ 5- 



..6' 



No 



G.' the written style of "Assess- 
ing Re^purces and Constraints" 
adenuat"e-?. 



•3 



5 



Yes 



HI. Value of the Module 

A. Do you feel that Module Four 
helped you identify and assess 
resources and constraints that 
could determine the feasibility 
,of an. instructional program? 



B. D'fd you like this module? 



2 
2 



3 4 
3 4 



5 
5 



6 
6 



J 
7 



No ' 
No 
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C, Are the resources ^nd 
constraints identified in 
this mocfule similar to the 
prevai:ling ones in your own 
^hool or' district? * If not, 
please list below the ofies 
that are not.v 



If you were engaged in designing 
instructional programs for your 
school could you use'thjs module 
to guide you in assessing resources 
and constraints? 



Mean 



NOT At' 

ALL' . MODERATELY COMPL€fELY 



' 4 



6 



Standcad 
Achieved? 



No 



No 



IV* Take-Home Exercise 

A. Are the Stmulated Case Study 
and InformatiT)n Bits on 
Janus Junior High adequate 
for establishing the context 
for the Take-Home exercise', 
on resources and^ constraints? 



B. Does the set of Curricular 
objectives adequately define 
the Environmental Studies 
Program so that you can assess 
the resources and constraints 
^verrLing it We 1:1 enough for 
you to judge its feasibility? 



1 2 3 4 5 6 7 No 

4 /■ . 



.V 



1 2 3 4 5- 6 7 No 



Can you identifyHhe prevailing' 
resources' and constraints on 
the. Environmental Studies PrOgram 
given the Simulated Case Study 
and Information Bits? 



No 



IS 

3e 



r . ' ^ APFECTIVE.QUESTIONNAIRE ^ ^ " ) 

^ f / 

Pocket Curriculum Analyzer 

^ ' The Pocket Curriculum Analyzer is a, tool .for helping you analyze, ..design^ 

rate,' and select instructional programs.. to.heTp"us determine its effectiveness, 

.please respond to the questionnaire by circling the.. appropriate number following 



each questiojfi. Please write in any comments that would further clarify your 



responses. 



I. Value of the Analyzer Elements 

A, Do you think that the Analyzer 
would enable younto design your 
^:own curriculum even if you had 
'* not played Blackboar;i Blues? 

Does the Analyzer help you " ^ 
organize your thoughts^ about 
an instructional prp^am? 

C. * Does it help you^tablish a. , 
set of preferences for youu.'pwn 
instructional program? 



Mean Star:dard 
NOT AT V, ^ ' Adhieved? 

ALL MODERATELY COMPLETELY ^ 



3 .4 



w 

5 



4 5. 



No 



Yes 



No 



Do you think the Analyzer 
-will 'rate different curricula 
well" enough to enable you to 
compare them and make a 
selection?. 



1 2 3 4 5 



6' 7 No" 



Does the "weighting" system 
refine the selection process 
to the point of enabling you to ' 
'seflect a program among -seemingly 
equivalent programs? 



1 , 2 3 4 5 



6 7 ;jo 



II. Methodolo(|.y and Execution 

A. Are the directions for .using the- 
Analyzer cigar? 



2- -3 4 



No" 



is, the. Rating Systfefn and are, 
tHe decision-points useful? 

'■ ■ / , 

C. Is the scoring systetti decisive 
enough fio yog think. that the 
weighting, system wi 1 1 be- tised 
too often')? 



' • ' * - Mem 

^ NOT At • ■ • ■ 
" . ALU (WDERATELY -COMPLETELY 



1 2;* 



v 



St.cmdca>d 
Achieved?, 



No ^ 



No 



Are the design^categories in 
the^Analyzer similar to the 
ones your school normally 
""considers? If not^-^please 
list them below: 



'J 



E. Are the categories organized' 
in such a way that the Analyzer 

*- is a gnified and selfi-contaihed 
urfit? ^ " ' 



1 2 -3 ' ■ 4 5 » 6 7 



1 2 3 4 ^5-5 7 



No 



No • 



Is the written style of the 
<»Analyzer suitable for this 
sort of tool? 



V 

4 5 



6 - 7 



No 



Value^-of ,the Analyzer 

A. Do you feel that the task ' ' 
of X rating and then selecting 
instructional programs are 
in^diate to your profes- 
sional needs? 



1 "2 3 



■ 4 5 



6 7 



>vNo 



B. ,If you were given the task 
of designing or selecting an 
Instructional program for your 
school, would you use the 
Analyzer? 



1 2 3 



4 5 



6. 7 



No 



If yoti used, it to rate several 
curri'cula, would you relj^ on the 
rating and/or weighting scores/ 
given by the Analyzer?- 



3 ^4' 



No 
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■ "^Ssx^ General Comments - Written and Verbal * 
•Hodula One-T "Blackboard*.Blues" Game " , 

1. "The pre-test at the beginning was too long -and difficult ta imdersland:" 

2. ''t)nly present game in one three ^tH- session"; nothing else." 

3. "I feel that you should be ^ble to make more than one choice per step."*!), 

4. "Th4s game is a good warm-up exercise for orienting teachers .to the diffi- 
culf^process of curriculum development.^ It is.very c^)mprehei?sive in'al^ 
areas covered in the area of curriculum design. Provides much informa- 
tion and is comprehensive in scope." f ^t' ^ 

5. "It is too easy to becpme detached from the pucpose of the game and make 
a 'game' of it. ^There. should be. a way to make players commit—they have 
to feel a responsibility even' if only intellectuany induced." 



"r'oderator should be stronger in keeping the princiole of the game before 

■ * ■ ' - : G ' - ' 

the players—What is the object?; is what you are doing contrary ar in 



Vine with this?" 

7. "Wirvning the game is not enough. The^e must^be a feeTing of creative 
pride in the design itself that seems superior to the participant, for 
-a reward." - ^ - * ' 

8» "Organization anci coptent ^f the Game board was excellent." 
9. "Game definitely helped me gain an understanding of the design process." 
10* "I liked the module even with the shortcomings! (not enough time)* Under 
better circumstances I would definitely like it." 



r 
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General Coinmen ts - Written and Verba 7 
• Module Two - "Analyzing. Basic Assumptions" . ' ■ 

1. "Interesting exercises; however, fails to have* the reality and'urgency 
^ facing administrators each day." 

2. "I /eel adrift--and haven't internalized the situations." ' 

1- li'ked it because it emphasizes the importance of analyzing the value. 
*scope, purpos&^^and nature of the educational process.". 
•4. "A good.. critiV^l analysis techniqup designed,/in a very^joyafale format. 

5. "More time required to do this type of activity when there has been no 
previous experience in this process." 

6. "This was difficult at first, not because of^the material but because of 
njy o//n 'set' which had turned me off." * ' ' ' 

7. "I found little evidence of a process of analyzing basic ,assumptions ex- 
cept to go back and. check evidence." • . 

' ' ' 
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General Cominents - Written and Verbal 



f4odule Three - "Matching Programs With GoalS*^ 
>• "It was much clearer than Module 2.'* 

2. "Length of Module 3." 

3. "Too much 9 too soon." 

4. "If I were a full time curriculum speci^ist instead of a classroom tedch- 
er, the. material would be most beneficial." - V 

5. * "Much too long and too much writing • Discussions v/ere good--ta^d reading 

v/ould ^enhance this program. Some of the information was helpful > however. 

6. "Too much at one time without motivation, and in general it was not mean- 

/ ' 

ingful to me." / 



I 




/ 




(A 



General C-omments - Written and Verbal. 
Module, l^our - "Assessing Resources- and Constraints" - 

1. "Latter part of module. not specific. Ask^^for very fa.ro'ad statements." 

2. "fiood exercise, in analysis," 

3. "I can't even remember it at this point— too concentrated an approach 
on top of an already crowded sch'edule^which was typical of niost of the 

\ 

Students," 

4- "It v/as better than the others because it v/asn^t so lonql" 

5. "Definitely could use thi.s module as^guide in assessing' resources and 
cons'trairits in my school •" ^' ' ' • 

6. "Resources and constraints very simvlar to the prevailing ones in my 
' school," / // 

7. "This is a fascinating adventure in curriculum development; however I 
feel' that your "modules" could very well be a semester course in itself," 

8. "Morale's too low to give a true assessment of your moduleJ" 

9. "Suggestions: 

1, Present this workshop at early stage of course. 
i. Clarify with professor the work expectancies—so he 
matches his assignments yjespectively. 

3, Clarify with students task expectancies prior to tasks. 

4. think the materials are generally good and useful-- 
provided p.resented-"nn a^good setting re: . , 

a. attitudes 

b. . timfe allocation 

c. take-hoI^^e work expectancies." 
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General Comments - Written and Verbal ■ 
fackel Curriculum Analyzer j ' • 

'I. "ram thoroughly confused." ' 
,2. "The idea seems to be Iseful, but the directions are totally unclear-, 
•making the whole experience negative." ' i ' 

\* — 

3. "The Analyzer serves as an excellent source in evaluating one's program 
for specifics. It is well thought out. The Curricu-lum Analyzer enables 
one ta double-chefck his own instructiopa^ogram (or daily lesson plan)*" 

. for consistency in objectives, methods, and materials used toreach^the 
desired^outcomes that are consistent wjth g^ls earlier voiced at the 
start of the school year." . . ^ 

4. "Directions poor—task poorly defined-di sections confusing." 1 

5. ' "I'm still 'confused ar^to how it operates." ' - 
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OCTOBER 197^ 



Abstract 

SVALUATIOSr REPOBT 

■ON THE TI^INING UNIT ! 



DBSTGI^NG INSTRUCTIONA L ^PROGRAI'lS 



Six training situations- similar to the normal "use 
marlcet«-f-oE-jthis training material developed by the Par 
West Laboratory for Bducattonal_Hesearch and Development 
were involved in the final field test durIng~^h<L.Sprlng , 
and Summer of 19?^. Analyses of. the dat^ obtained indi- 
cate positive affect i!ve responses and pra'otically signif- 
icant iii^reases in the ability of participants to identify 
important curpioulum comparison ca^gories"; . to specify 
appropriate classroom procedures given a. set of cogni^ve 
and affective goals, to calculate rough* ois at analyses of 
proposed instructional programs, and to utilize the sped- 
fic vocabulary and_ concepts of curriculum design of the 
unit developers.' A distribution version with coordinator's 
handbook is currently available. . It is reeommendsd for use 
in 'college and |chool district teacher education sittings. 



Valuation Report' by 




_^ E, Temp 

Research Psychologist 
Lafayette, California 
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INTRODUCTION ' 

The ne§.d for Evaluation of newljr developed oul^ric-^. 
ular materials is now widely accepted* Therefore, this 
report wili not dwell upon the importance and urgency N^f 
such 'evaluation studies* The training unit Designing 
Instjguctional Profi^rams ^ developed by the Par West Labors* \ 
atory! for Educational Research and Development, was eval- 



uated 1 during a. final field test in the Spring and Summer 
of 19/j4. This report describes the materials and the in- 
formation base upoii which evaluation conclusions were made. 



and suggests the most likely cognitive and affective out- 
comes to be expected in future uses of the distribution 
vei*sfon of the unit. 

Description of the materials * ^ The distribution ver- 
sion of the training materials consists of five, modules 
(bound in two unequal size volumes), the. game materials* 
^necessary for playing Chalk TalVr ^ and a coordinator's hand- 
book* ^ 

According to the developers the unit's overall pur- 
pose is to improve instruction by preparing school decision 
makers to make more defei^sible choices regarding the fom^ 
and contfent of instructional programs • To achieve this 
overall purpose each module has more specific purposes. 
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These purposes are listed beiowi 



to: 



The purposes of the Chalk Talk game (Module One) are 



provide the vooa^bulary and definitions requisite 
for designing instruct ional^-programs 

providd* a procedure for making systematic deci- 
sions abs>ut> the;, content of programs 

provide a mearfs of analysing program decisions 
for logical. consistency so that the effects of 
initial decisions can be traced in the final 'pro- 
gram outline 



/ 



provide a vehicle that familiarizes participants 
with making, defending, and'&^evislng program de- 
sign decisions in a group ^o^that each player 
whether student, t>parent, . or professional can 
understand the rationale and procedures beh^d. a. 
certain program design - 



The purposes of Anaiyzinjcc Basic Assumptions (Module 



Two) are tofj 



• provide an organisjing f ramework by which par- ' 
ticipants can examine their personal feelings 

^ al^out human beings, the^ world, and learning so 
' ^ thai; decisions a^bout the use of particular in- 
structional progrem procediires can. be made' in 
light of these feelings/ ^ S ^ * 

/ • ^ provide a means by which participants can an- 
alyze their ^assumptions'' about people and learn- 
ing in relation to the assumptions of others 

• provide a vehicle through which^ articulated 
basic assumptions oans^ agreed' upon, t9 estab- 
lish a group value basis on which programs can 
b,e designed. i • « 

■ • ' ' ' ■ • V' 

• " enable the most approprlatei program procedures 

to be selected oni the basi-s^ o.f consensual lists 
of basic assufiiptior^s about learning 
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The purposes of Matching;^ Programs With Gouls (Module 
iree) are to:. . ; 

p2X>Y4-de significant characteristics of; various 
/ ^ types :of goals so' that goals can he categorized 

^ , • provide a tool ;snd guidelines for sorting dis- 
parate ;goai*",statements into cqmmon types 
< ' ' k. < » ^ 

• / provide a; tool. and gul^^lines for classifying 

observable; or measureable behaviors so that a ' 
startih^^ppiht^iniay be reachssd for .matohing pro- 
grams \jrith selected ,goals 

. * provide a meanff' flor developing a complete pro- 
,gram o:u1iilh'e thlat ^^.s con slstenjt, with pre-stated 
• in's tract ignal goals and objectives 

.Tlri© purpdses of Assessing "Resources and Constraints 
(Module Pour) are tot, ^ 

• provid.e J.nstruments\8o th^a^ p^axtioipants can 
/conduo|; r^ugfi- co^t analyses .for proposed .pro- 

Kra^s^ ^p^^ o*i?.3^^s oJT, staff req^uirements , materials , 
- ' faclilt'ie8\andv^qulpmeht; ' : 

• .pro:^l(ie.:a, pfb^ partylcipants to ratei in- 

structional ;*p:^ogr .feasibl'lity by '•.tra^l'ng-i* 

oi*f" certain »aspec1;s of programs for other, more 
desirable ohes , i ' 

• • provide examples of institutional resources that 

can help make programs work, and constraints that 
should be removed or ^re unavoidable 

provide a procedure ^y which participants can iden- 
tify social resources and constraints that benefit 
'or impair the feasibility of instijucfcional pro- 
grams 
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The purposes of Selecting Curricula (Moduli Five) are 



provide a means for organizing partlclpJsints* 
thoughts about curricula for possible inclu- 
/ sion in program outlines 

. provide a procedure s$ that information about 
(• curricula cari^ b% gathered and recorded in a 
' systematic manner , 
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enable participants to Tate several alternative 
ciirricuia for apprcpriatfeness in their schooj;^- 
setting . ' ' ^ 

• provide a procedure so that curricula* can be 
' rank-or.d?red accordina to preferences, and re« 
^ quiremeritsi of outlined programs 

The trainlTtig unit, in total, attempts to accomplish these 
purposes by providing participants knowledge, Sidelines, 
and tools, in a series of 'flexibly administered wdjpkshop ^ 
settings that are guided. by' directions and suggestion con- 
.tained in the coordinator's handbook. 

This brief description of the loaterialis should.be 
supplemented by detailed examination of the training unit 
itself in order *to help the reader envision the currioular 

experience evaluated and reported on in this paper. Ideally, 

* ■ "* 

anyone planning to use the training materials In a college 
■ or school setting would f ir^t go t)irough the experience as a 
participant "under someone else's direction and guidance. 
' However, it is clear from the field trails* C9mpieted that a , 
coordinator cai? do an excellent job Just by using the coordl- 
nator^s handbook and answering questions by oommon sense as they 
arise, during the training sessiohs*, % * 
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UA-BION- METHOD ■ 

1- ■ ■ . •■ ■ . 

De script f on pif the e-vjaluation t>lan . There were two 
main data collecting actlvlltl-es built Into the overall' 

evaluation clan. These were (1) direct observation of the • 

\ ' . P ' . . 

conduct of a sample of the If 1 eld trials, an^ (2) paper and'_ 

pencil tests administered als PREVIEW and EPILOQUE Instru-' 

ments. ' . ' 

Dlre.ct observation was \concemed with (1) noting the 

^oblems of the Coordinator knd participants in dealing with 

the training Materials, (2) questions that were raised by 

participants that indicated confusion or misunderstanding of 

« 

the dir,ectior.s or concepts in the materials, and iS) affective 
reactions to different portions of the scheduled activities 
or fatigue reactions to the intensiveness of the trainings 
The purpose of tlie direct observation was to collect forma- 
tive, kinds of information whibh have been, reported separately 
t.o t^he developers, and to aid in interpretation of the ob- 5 
Jective measures of learning outcomes collecte.d by the paper 
and pencil tests*' 

Description of the data gatherinp; instruments . The 
PREVIEW .and EPILOGUE instruments were designed and revised 
by the .evaluator. These ItjiStrUments were designed to collect 
as much- information as Jjosslble in~:a' restricted time perl.pd. " . 
'it was important that the 'time spent on testing not be con- 
sidered as intrusive or unreasoriable by the volunteer trainees 



' TKe instrument's" finally developed required 25 to 40 
" minu\es ^br 'completion, with S;he EPILOGUE instrument re- 
guirini? close?r;to '40 minutes because of the inclusion of 
two^additiqnal questions. 

Jh.e'a'ssessment situations in the evaluati9r. instru- 
ments were related 'to general skills and abilities useful 
' . to anyone working on the* design- of instructional programs. 
That Vs/the test situations could bo used independently 
'^rci^-the training materials to arrive at conclusions about 
a group's, present ability to under^e some typical currlc- 
^ulum committee tasks. - » 

The. three- important abilities assessed were: 

th^' ability to select curriculum comparison 
^ categories of importance 

. the ability tb' specify appropriate class- 
* • room procedures to accomplish a given set 

. • of behavioral objectives 



the ability to compute rough coat analyses 
of a proTDosed ."prSgram given a set of d^- 
trict' guidelines and cost figures ■» 



• ' 'The tt^aining-ur^lt being eValuated was not the only 

. experi<?nc4 thaj; might have incrV^^d these abilities in ■ 

-participants. For ins^ance^-straight lecture approach. 

on these topics- might hg;ve /produced eithe§^reater or V 

le'sser -feains i^n these abilities. The assessment situations 

w-re designed -to allow^any increases in the group's abili:.ty 

to deal with comple:f situations % .emerge and be shown. 

v.\ V ' / 
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In addition/' the EPILOGUE instmment allowed for the 
collection of information on two curriculum specific 
questions that could only be answered by persons exposed 
to the ^ppbific training materials being evaluated* 

Both the PREVIEW and EPILOGUE, instruments contained 
a section designed td»obtain information about .the more 
-^^ subjective and affective reaction of participants to their 
experience, and to their sense of being able to perform 
well in a curriculum committee situation* 

Appendix A contains complete copies of both the PREVIEW 
and EPILOGUE" Instruments and should be examined In detail tsj 
those interdrsted In the specifics of the assessment situations. 
Some fueling for the assessment's Ituatlons Is also contained . 
in a later section of this reports' Evaluation Results and 
, Conclusions, where the specific items are discussed* 

Description of the Final Field Test situations . The 
Pinal Field Test series consisted of six trials conducted 
from Hay through August, 197^. The first in the series, the 
' May trial, was a pilot test of the'prpposed field test ver- 
sion of the materials and of the evaluation instrumentation 
and strategies* , Revisions weire made in both the materials 
'and the evaluation instiniments on the basis of the pilot 
test in May* ^ 
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The remaining five ^^^^^ » ^^^^ during July and 
August,' provided the information base upon whibh evalua- 
tion conclusions wer^.e reached. It is apparent, however, 
fxom the items that remained unchanged in the evaluation 
instruitfents that the pilot test situation in May would 
have fitted" comfortably jfithin this final series" If the ^ 
total test items had remained unch&nged. That is, the 
six trials all reflect essentially the same kindSfiOf cog- 
nitive and affective^out comes, although ohly the final 
five t;pdals are reported on .here. 

>- 

Summary of Pinal Field Tep*^ _ 

May t-f'iai (pilot) N= 10 teachers and administirators from 

Catholic school system 

July trial . N= 26 college class for educational ad- 

ministrators and persons planning 
to be administrators 

Augiist trial (a) N= 8 special one credit college course 

held during one week- 
August trial (b) N= 7 ixi-sjirvice scttqol district retreat 
... for administrators 

August trial (c) N= 19 in-service school district workshop 
August trial (d) N= 1? school district pre-Pall workshop 



The-itotal number of ^ participants in the five trials 
that form the basis of this report was 77. This includes 
two trials in college class situations a)id three in school 
district' use situations. This distribution reflects the 
expected two "use markers" for the training unit. 
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^ .Organization and conduct of a field trial, Just as 
any future use of the training materi&ls, was structured 
and guided by the coordinator's handbook that accompanies 
the traiiiing unit* ^.Qf the final field trials reported 
here, three of the trials were led by the developers;^ two 

s 

of the trials were conducted by others, unfamiliar with 
the materials until the actual field trial and using the 
handbook as guide. 
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EVALUATION RESULTS AND 
CONCLUSIONS . 



This section of the evaluation report details the results 
obtained and. the conclusions arrived at point by points Al- 
though an' effort has been made to make this. as clear as possi- 
ble for the reader some consultation vrlth the evaluation in- 
struments. in Appendix A will aid understanding greatly. This 
lengthy section of the report is divided into cansideration of 
cognitive outcomes and affective outcomes separately • 



Coprnitive outcomes* One of the assessment situations 

r . ^ 

presented to the participants, both before and aifter train- 

/ * 
Ing/ was introduced in the following manner: 

Situation 

i The following areas or dimensions for comparisons 
^ among curricula have been suggested by a sub- 
committee of the District Curriculum .^Committee. 
Now it is time for each member of tHe""* Committee to ^ 
vote for those categories of comparisons that are 
absolutely essential if the Committee is to do a 
- really "expert^ Jol?* On this and the next-page, 
7 please vote by* checking tho$e categories! of com- 

pari sons that you feel ought to be used to compare 
different proposed curricula* ^ 

^ ^ * iBy examining those categories of comparisons most 
frequently selected by group members, It Is possible to 
describe 'the 'criteria the group believes are important. 

On the PREVIEW, 8? per cent of the participants re- 
ported themselvek ^as having had advanced work or degrees 
- in curriculum and instruction or other education special- 
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ties. * Therefore, these -participants were not tmtrained 
or unfamiliar, with curriculum analysis concepts ^vPrlor to 
working on Deslfl:nlng Instructional Programs . 

Result . Prior to' training on the unit, 90 per cent 
or more of the participants considered as "absolutely- 
essential only two dimensions of curriculum comparisons. 
These werjeJ • ' ^ 

' . scope of the content covered 
•Intended results of instruction 

, . — 

If a criterion of 75 per Pent or .more were used Instead, 
than three additional dimensions would be added. These weres 
. organization of the content 
. reading difficulty level of -the materials 

/ . goals and objectives stated for the materials 

Together these five aspects of comparison among dif- 
ferent curricula make a respectable but rather restricted 
set to be considered in selecting a program for adoption in 
a school district. 

- - - J' ' 

Many of the piirposes. and objectives of the training 
unit, in the opinion of the evaluator, were aimed at in- 
creasing the ability of participants to select curriculum 
comparison categories of import^ce. If the training was 
effective in this major area, then we would expect to see 
a much more comprehensive set of dimensions endorsed ^ after 
exposure to Desiffiihg Instructional Programs ^ 
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Such was the case. After training a greatly expanded 
j.ist of d.-....2nsions were selected by 75 per cent or more of 
the. participants. Besides the five dimensions already listed, 
the group now indicated that the following additional aspects 
of the proposed programs ought to be compared J 

• f imctlon of the teacher -implied in the materials 

« role of the student implied in the materials 
(passive/a^ctive ) 

. teaching/learning method implied or stated 

• size of student groups Inherent to the jnaterlals ^ 

• special requirements for classroom' staffing 

. necessity for any speolali:ied equipment or 
hardware 

physical arrangements for use inherent to the 
materials ^ ^ ^ 

. special subject matfer competency required of 
teachers using the materials 

cost per stMent to purchase and maintain these 
materials 

• kind of student evaluation built in (exams, no 

exams, etc» ) ^ . - 

• length of the. complete curriculum 1^^ weeks or 

semitosters 

. educational philosophy implicit or stated 

n 



A total of 1? comparison categories were felt to be 
important by the participants after training* Many readers, 
studying in detail this extended list, might agr^e that a ^ 
curriculum committee that did its work considering such things 
-as costs, evaluation, special staff requirements, and so 
■forth would "be more likely to do a better Job. 
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Gonclxislon ^ A practically significant expansion of the 
ability of the participants to select curriculum comparison 
categories of importance was achieved* The gz*oup that prior 
to training would have been content with comparisons on two 
to five dimensions was, aft^t^training, indicating that up 
to 1? comparison dimensions were essential to doing an ex- 
pert Job# ^ ' 

X 

Copgiltive outcomes (continued)* Another cognitive 
outcome of jimportance, and central to the main thrust of ^ 
the training materials, was assessed in a complex situation 
to be found on pages 4 and 5 of the PREVIEW instrument in 
Appendix A* This assessment situation measures the par-^ 
ticipants' ability to specify appropriate classroom prdoedures 
given a particular's at of cognitivev and affective goals to 
be accomplished* 

The five behaviorally stated objectives to be accomp- 
lished (f^r a Spanish program in Grades 7-8) were studied 
by participants* They then were to indicate those classroom 
procedures and conditiore which most logically should be' 

followed to accomplish these objectives* v 

^ J 

P.esult * Prior to training when asked to outline a 
preferred method to accomplish the set of o*bJectives given, 
50 per cent or more of the trainees selected the following s 

• a laboratory/practical experience approach 
to teaching 

* the teacher to function as, a diagnostician/ 
prescrlber and resource person 
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• ^the stud^snt ,t^o function as a .mutual plsuiner/ . 
performer ' \ ^* ^ i 

• e-^aluation tb be a Joint function of teacher 
and student 

• materialsrused to accomplish the objectives 
would be programmed materials and teacher/ 
student prepared materials supplemented with 
recorders ^ ^ ' " ^ , • 



• instruction would bfe held 'in a single class-- 



room with movable furniture 



Thus, in effect,,^the gix)up outlined a logical and 
appropriate set of classroom procedures or conditions to 
accomplish the given set of goals* This should not be 
surprising because these all experienced and trained 

educators used to making" decisions ibout ^"such matters. 

Before trainihg, thes^ same pe^ople had expressed 
their greatest amount ofc, confidence as a group (62 per 
cent ) that they had the ability' to do Just that t specify 
appropriate classroom procedures, given a particular set 
"of cognitive and .affective goals* .This wQon^-trasts with 
their 16 per cent expression of confidence that they had 
the ability to clearly define critical characteristics of 
different types of learning goals;' their 26 per cent 
confidence in their ability to compute rough cost analyses; 
and thoir ^2. per cent confidence that they could classify 
curricular objectives using Bloom or Krathwohl* Only in 

i 

confidence (57 per cent) about being Able to state personal 
assumptions a.'bout man and ^.earning did they ^approach the 
level associated with their ability to specify appropriate 
Classroom pro*cedures« 
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Given that th^y had confidence that they could 
specify procedures and that they did outline a credible 
program (within the limitations of the test question )9 
what kind of 'growth, if any, would it be logical to ex- 
pect on the EPILOGUE presentation of this situation? 
> 

Because there are many acceptable and , appropriate 
classroom procedures to accomplish an^ set of behavioral 
outcomes, there are many "correct" solutions. ^ Thus, dlf- 
f erent observors might validly be looking for shifts toward 
their ovm favorite '^correct*' solution. What was found was 
that some significant additiqns to the set of classroom 
procedures were made and one major shift was introduced 
after training. 

J* 

The additions were (1) broadening the teadhing/leamlng 
method to include an emphasis" on memorization (it was a foreign 
language program), and (2) widening the conception of the ^ 
teacher's function to include the role of information pur- 
veyor and the student's function to include listening/ 
following. 

I 

These additions appear to be worthwhile and more 
r,ealistic to the evaluator. . ' 

The shift was from emphasis up6n programmed materials 
and teacher/student prepared materials to standard texts/ 
workbooks and audio-visual materials to accomplish the set 
'of objectives. 
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Conclusion ^ A more realistic and wider set of class- 
room procedures and conditions were suggested by the yartic- 
ipeuits after exposure to the , materials and experienc^^s of 
the training unit. Some significant improvement in the 
ability of these experienced educators to specify appropri- 
ate classroom procedures was found. 

Cognitive. outcom> s (continued). The third main ability 
assessed *in the area of cognitive outcomes was the ability 
to compute rough cost analyses of a proposed program. The 
situation is summarized below: 

Situation 

It is difficult to talk about new instructional 
programs without considering costs. In this meet- 
ing of the District Curriculum Commitlxpe ^e want to 
do a rough cost analysis for the nevjly proposed 
Career Education Pp^ogram. The Sup~erintendent has 
indicated that $10,000 is available for this pro- 
graiu# We need ta^ tell film how many classes of 
students (average si2:e^ 32 students) could be served 
^ i by this program each semester. He has provided us 
with the following budget Information: 

(Budget allocations and cost figures 
provided here^ Also a Cost ■ Analysis 
Form to be completed. ) 

This six;uation^ altho\jgh perceive., by the examinee 
^s a calculation problem, 'is used to assess the ability of 
the person to understand the process of allocation of the 
money resources available. Th^^^arithmetic invdlved for 
someone who sees the underlying ra;tionale of the cost an- 
alysis form and allocation percentages provided is quite 
simple. Minor arithmetic errors were not of concern if the 
total set of answers given by the trainee seem to reflect 
understanding of the allocations to be made. 
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Result . Prior to training on the imit, 27 per cent 
were able to complete the cost analysis problem correctly 
or partiallyj correct* This figure is interesting because, 
as reported earlier in this paper, some 26 per cent felt 
that they had the ability to compute" rough cost ardyses 
prior to training* This Indicates an awareness on the 
part of these participants that mathematics is not one 
of their stronger abilities^ 

Much" of the work in Module ?o\ir. Assessing; Resources 
and Constraints * deal? with the completion of. similar 
cost analysis forms and discussion of trade-offs to be 
made when budget f igtorSs^^iiow* If the training was ef- 
fective in this important area, it. would most likely be 
shown in an increase in the ability to see the logic behind 
a budget forecast form rather than in any real increase -in 
ability to do simple multiplications and divisions* Of 
course, such an increase must be inferred from the ability 
of the participants to compute roUgh cost analyses that 
baffled the large percentage o^ them "Just prior to train- 
ing. 

• I 

The after training results support this idea* Upon 
completion of training 61 per cent of the group were able to 
correctly or partialj.y correctly complete the cost analysis 
form* There were also fevfei: strike overs, side calculations 
and minor errors on the EPILOGUE copy of the cost analysis^ 
form than there had been on the PHSVIEW pages* 
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Observation dixrins administration of t.he' EPILOGUE 
evaluation Ihstrumertt als6 supports the idea that far 
fewer of the participants had real difficulties in conl- 
pletloxi of thl's task* * On the PREVIEW many participants 
had failed to do the Item at all even- after extended study 
of the problem and even some tentative calculations* 

Conclusion *^ Many mdre of the pear.tlclpahtf under- 

stood the logic of a cost analysis form and wete able to 

compute ro\igh cost analyses .after ^training than before* 

A practl'cally significant increase in the ability of 

participants^ to do such analyses of a proposed instruc- 

tlonal program was apparent, . - 

\ 

CoCTltive outcomes (concluded). Because practically 
sighif Icafit increases in abilities of participants in the 
three major areas assessed were found, there is less inter- 
est in examining the lower-level knowledge outcomes* Thes 
outcomes^ concerning vocabulary and curriculum specific 
items, were assessed by the evaluator in casfe^no larger 
and .more important gains were made by t^e trainees. 

Briefly, it was found that a majority (53 to 62 
per cent) of the participants could demonstrate recall 
of the critical characteristics of the four different 
types of learning goals stressed by the developers 
of the xmit* And 22 per cent could give a 'full or ^partial 



answer to the question: how does analyzing our bSsiV as- 
smptlons about man Glnd! leai^ing relate to designing 
"good", instiructlonal programs • 

Both :of these items, were design to mear^ro, straight 
recall of Informp-tion presented in the training program. 
Correc^i^ answers had to rei5roduce the information available 
in the materials. ' Since such -recall was not stressed in 
the materials or in tKe instruction, It was not expected 
that most' participants would be able to do very well in 
these questions^ In a real sense such feats of memorizar 
tlon are not required of participants because the materials 
are dasigned to provide various "tools" that the trainees 
carry away -with them for future use. 

The results indicate that if tl^iifelt developers do 
want to accomplish knowledge/recall objectives (some of 

;vhich are stated in the materials).', they will have .to 

*■ 

adopb a different and iSethaps more traditional memorization 
and study for examination type" teaching strategy. This is 
not compatible wi*,h the developers more over-tiding ob- 
jectives of achieving application type skills in the shdyt 
training period provided, ^d' is j^ot recommended by the 
evaluator. ^ , ' 

. \ ■ 
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. * Affect i.ve outcomes » . Subjective reactions and comments 
about the training unit were collected in two ways: in 
x^riting and orally* The written portion was included in the 
' EPILOGUE instrument and the oral loortion was collected by the 

eyaluator while present at the training sessions* 

The first written question asked was: 

VJouid you recommend the training experience just 
completed tb^o.thers? 

P-artlcipants were asked to mark: Yes, No, or Uncertain. 
A space was allowed for aiapllfylng comments under this qu,es- 

* « 

tionv " ' • : ■ 

^ Summary of Results 

Would you recommend No -^^ 0^ 

the training exoer- 
, - ience Just completed Yes 88__ Per Cent 

to others? 

Uncertain 12 
The second written question participants were asked 

was: 

Would you utilize these training materials ■ 
with a curriculum study of which you were 
selected leader? 

Again participants were asked to check: Yes, No, or 
' Uncertain, and a space was set aside for comments* 

A'- •- 
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Summary of Results 



Would you utilize these No _.2_i,inclucLes not 

training materials with applicable) 
a curriculum study of Yes 81 Per Cent 

which you were selected 
leader? Uncertain 17 



Although it is clear that the overwhelming response 
of participants is positive, both the amplifying written 
comments and the oral comments collected by the evalua- 
tor were considered also in reaching a conclusion. 

Some of the negative 'comments used to explain an 
uncertain rating are quoted or paraphrased below* 

. Time frame maybe unrealis\;ic 
. Seemed rushed — • noisy -~ 

• I am concerned with imclear -terminology, direc*- 
tions, .and hassellng with the mechanics of 
using the materials - ^ . 

« I believe that I need mpre training 

t 

Positive reactions voloxnteered by the participants 
Included: 

• It gives a definite starting place in 
curriculum development 

• These toodules lend themselves to systema- 
tic teaching of teiachers.*,,the process is 
very relaxed; friendly 

rf I would like to set up a college credit 
in-service program where participants would 
use the modules and then a'pply them to an 
actual currlcular problem 



ERIC 
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Direct observation and discussion with participants 
by the evaluator both formally and informally support 
both the enthusiasm of the participants for the experience 
and their statements of intent to use the materials if the 
actual opportunity arisesi. It was clear that the vast 
majority of those in the Pinal Field Trials were highly 
satisfied with the experience. The only recurring complaint 
was about the intensiveness of the experience and the need 
for more time to fully comprehend what had been presented. 
Many recommended various ways to expand the total time de- 
voted to- the -training unit in future uses. 

Conclusion , In the area of affective response to the 
training experience, participants were demonstrably posi- 
tive and enthusiastic about the unit, and they were partio- 
ularly desirous of increased time for the training ex- . ^ 
perienoe. The training materials were successful in arousix^ 
interest and appreciation of the concepts presented. 



71 



- 

SUMMAar AND RECOMMENDATION 

» 

The Pinal Field Trial of the Desiring Instructional 
Programs unit was conducted during the Spring and Suimer 
of 197^» The evaluation results indicate parytlcipants felt 
poitively about the training experience and that the;y 
made practically significant increases in tiielr abilities 
to: . I . 

(1) identify significant curriculum compar- 
ison categories; 

(2) spjscify appropriate classroom procedures, 
given a particular .set of cognitive and 
affective goals; 

(3) calculate rough cost analyses of proposed 
instructional programs; and , 

(4) -utilize the specific vocabulary and con- 

cepts of curriculum design employed by the 
unit developers • 

The body of this report has presented the evidence 
to support these statements and provided more comprehenr . 

sive specification of each of these conclusions. 

/ » ' ^ ' 

/ 

Recommgndafelon . This unit should be made available 
in a distribution version for use in college and school 
^district settings where teachers and other school person^ 
nel are in training foir curriculum design responsibilities. 
It is an effective unit for accomplishing the behavioral 
outcomes specified above. 
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I APPENDI}^ A 

i 



The EPILOGUE instrument contains both pre and post 
testing results. On items that are unique to tJhfe' PREVIEl-/ 
or EPILOGUE instrument the results are reported in the 
appropriate space on each instrument, resoectively. 



1> 



ERJC 



'Your Name 



Results reported In 
percentages of the 
total trial group 



PWiVIEW N = 77 . 

There are inany important skills and abilities that ivill be needed by 
anyo'' working on the design of instructional programs, Tlus training unit is 
about to exix)se you to a few of them in an ordered and interesting way. 
Before beginning that presentation however, it is necessary to collect some 
infonnation from you and to introduce you to the materials by a series of 
situations related to the training unit objectives. Please do the best you 
can on theses tasks now. You may begin as soon as you are ready, 

A * A A 

IVhich of tlie following groupings most closely describes your background? 
(Please select one that is closest to being accurate,) 

teacher training some advanced xork in 

curriculum and instruction 38 ^ 

advanced degree in curriculum ' advanced work or degree but not 

OlA or Doctorate) 19,^ curriculum and instruction 3q ^ 

1 ^ := not applicable (Supt^s secretary took course) 



If you were now requested to serve upon a Curriculum Committee, without 
further training, how confident would you be about your ability to undertake 
such an assignment? 

very confident somewhat confident not confident 



Place a check (\/) on the line provided below opposite each ability or skill 
that you feel you already, have, 

I have the ability to: 

clearly state my personal basic assumptions about man and learning 



EPIL0GU2 
for. all 
other 
results 
froM . 

for 

ease of 
comparison 



clearly define the critical cliaracteristics of different types of 

• learning goals 

compute rough cost analyses of instrucj^ional programs 

classify curricular ' 'ijectiv^es accordingnto a system such as Bloom's 

cognitive taxonomy or Krathwohl's affective taxonomy 

specify appropriate classroom' procedures given a particular set of 

cognitive and affective goals 

* * * 



/ 
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The following areiis or dimensions for comparisons among curricula have 
been suggested by a sub-cuininLttee of thu, District Curriculum Comittee,^ jNfow 
it is time for eaeli member of the Committee to vote for those categories of 
comparisons that arc al)solutely essential if the Committee is to do a really 
•'exj^crt" jol). On thib and the next page, please vote by diecking (v/) those 
cate<jories of compiirisons that you fee] ought to be used to compare different, 
proposed curricula." 



Categories of comparisons ' * ' 

number of pages of student self :^instructional materials 

scope of tlie content covered 

quality of the illustrations and graphics » 

intended results of instruction * • ' * 

function of the teacher implied in the materials 

social, etlmic or. sexual bias in the materials 

organization of the content , 

reading difficulty level of the materials 

role of the student implied in the materials (passive/active) 

availability of teacher manuals ^ 

teaching/ learning metJiod ijuplied or stated * 

size of student groups iiilierent to the materials 

presence of additional reading and study lists 

special requirements for^ classroom staffing 

grouping of students required by materials (age, ability, etc.) . 

quality of the v.Tit^ing in the curriculum 

necessity for any specialized equipment or hardware * 

special subject matter canpetency required of teachers using tlie materials 
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^' physical arrangements for use inherent to the materials- 

cost per student to purcliase and maintain these materials 

* kind of student evaluation built in (exams, nb exams, etcO 
reusability of the materials 

comparisorfs of Students built in (norms, class comparisons, etcO 

^ . availability' of answer manual for any problems or tests 

informatioij on how materials worked in classroom trials^ 

quality and durability;of the materials 

convenience of' the materials for student h^dling/storage 

goals and objectives stated for the materials 

consultant assistance provided in the "package"- 

length of the complete curriculum in weeks or semesters 

educational philosophy imfjlicit or stated 

^^availability of tests for materials 



Any comment: 



1 — — - 
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SriUATION 

A sub-conimittce of the District Curriailum Committee has submitted the 
following set of objectives for a new Spanish program for Grades 7-8 • 

Spanish Program, Grades 7-8 

All of tlie following objectives should be achieved by our students by the end 
of the eighth grade: ^ ^ 

• , • ^ 
1. The student sliould be able to translate correctly any written or oral 
'material described in the' Eighth Grade Cdurse of Study for Spanish, 



When' speaking Spanish to others in the class, students should be 
using correct 'pronunciation, and (j^ntonat ion, 

SN-Sfudents should be able to demonstrate coijiprehension of written 

passages read by the teacher by correctly answering questions (oral 
. and written) dealing with the content read, 

4. Each student shouldS^v^ mastered a basic vocabulary of approximately 
lOOC words of Spanish andjhe able to define a random sample, of those 
words in- Spanish, 

5, Students should be able ,to read a simple story in Spanish and 
converse' about it comfortably in ^^anish while danonstrating 
appreciative understanding of the ^wofy read. 

'* * A A 

* 

Given the achievements and beliaviors representediagst heavily in this 
set of objectives, please indicate by checking on the opposite page those 
classroom procedures and conditions whicli most logically should be followed 
by teachers in the schooL district. / 

First select the most logical choice(s) of teaching/learning method and 
then proceed to indicate your otJier choices within the other categories 
listed. 

Take time to' carefully study the set of objectives given before marking 
your choice of teaching/learning methoHsT ^ 



PAKTIAI. LISTJNG OF OPTIONS/qiOICES FOR SPANISH INSTRUCTION ' 



'I'eaching/Leaming Methods 

lecture/demonstration programmed instnaction memorization 

laboratory/practical experience discussion/seminar 

research/ synthesis discovery/inquiry 

Teacher Function " * , 

information purveyor diagnostician/prescriber contractor 

resource person fellow learner no teacher function " 

Student Function 

listener/ follower fellow learner self- instructor 

mutual' planner/performer primary planner /performer 

Materials 

standard texts/workbooks Selected readings - no materials 

, - programmed materials educational games ; A-V materials . 

teacher/student prepared materials specialized materials 

Physical Setting^ 

single classroom with fixed furniture single classrocxii, movable 

furniture 

open space classroom with moveable furniture and wails ' , 

library study carrels other in-scliool location 

out of school/industry location ' \ 

E quipment (noC materials) 

small hardware (TV, talking typewiter, projector, recorder) 

large hardware (responder systems with computer tie-in, machines) ^ 

specialized equipm'oniS (art, science, cooking,. etc.) 

no equipment needs 

Evaluation - 
— . ^ 

teacher evaluation of students student self -evaluation 

both teacher/student evaluate evaluation by standard tests 



mastery level- evaluation 



SITUATION 

It is difficult to .talk about new instructional programs without 
considering costs. In this meeting o.f the District Curriculum Committee we 
vv^ant to do a rough cost -analysis for the newly proposed Career Education 
Program. The Superintendent has indicated that $10,000 is available for this 
program-.-.. We need to tell him how many classes of students (average size, 
32- students) could be .served by this program each semester. He has provided 
us with the following budget information which is used in this District for 
calculating tlie annual budget estimates : 

Cost Figures fortOur School District 

* 

65% of any budget allocation goes into direct instructional costs 
"(teachers, textbooks, professional support, clerical help) 

351 of any budget goes to our fixed costs or overhead budget 
(i.e. non- instructional costs) 

Of an Instructional Budget Figure: 

801 goes' to teatlier salaries 

l'0% goes to professional support -salaries 

,,51 goes to instructional materials (texts, etc.) 

5% goes to other (clerical/secretarial, etc.) 

The curriculum nwterials for the proposed Career Education Program cost 
$10.00 per set. 

The average teacher salary is $10,080 for 36 weeks of instruction. 

The average professional support personnel salary is $7,920 for 36 weeks 
of instruction. 

Clerical/secretarial help averages $600 per month on a 12-month year^ 
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i J ' . ■ . 

The Cost Analysis For m (Oiar District Fonn 378) must be submitted with our 
answer to the two questions at the bottom of the Form, PleasQ do whatever * 
calculations fin round numbers) that are necessary to arrive at the .answers 

requested by the Superintendent • 

.1 

COST ANALYSIS FOI^l (B78) . 

!• Tentative Total Allocation, • $ . 

. V ^ (Obtain from Superintendent's Office) 

**Proposed Instructional Budget 

Teacher Salaries \\ $ 

ProfessjLonal Support Salaries $ 

Materials $ 

Other Expenses • $ - 

**Prop.osed Non- instructional Budget Overhead Costs, $ 



(Note: In order to secure final approval of tliis Budget the person 
submitting (or committee) must answer the following two questions: 

1. If this Budget is approved, how. many sets of new curricular 
materials will be purchased? sets 

2. If this Budget is approved, how many weeks. of teacher ^ 
instructional time will be charged to this budget? weeks . 



Summary 

Ffow Diany classes of students could be ^scived by this program each semester 
(normal teacher load, 5 classes and 1 prep)? . 

^ classes for > semester (s) 

@ a total cost of $ . . 



^All results are reported in percentages of the tot^l trial group* PREVIEW 

results are reported first, followed by EPILOGUE results. ^ 
YourNcuno ^_ 

EPILOGUE ' U = 77 . ^ 

Place' a* check iV) on die line provided below opposite each ability 
or skill that you. feel' you now have. - 

I have tlie ability to: 
PRE POST 

57 7^; .c learly state my personal, basic assunptions about man and learning 

4 

16 6U c learly define the critical characteristics of different types of 
learning goals' , » - 

26 * ' 71 compute rough cos,t> analyses of instructional programs 

'^2 68 c lassify curricular objectives according to a system such as 

Bloom's cognitive taxonomy or Krathwohl's affective taxonany \ ^ 

62 ^ 77 s pecify appropriate classroom proqedures given a particular 
^et of cognitive* and affective goals. 

* * A A ^ 

Would you recommend the training expex^ience just completed to o.tliers? 
88^j y es ' 110 1 2^ u ncertain 



Comment: 



Gonments reported in body of report 



Would you utiiize these fVaining materials with a curriculum stud^ i 
of which you were selected leader? 

81^ y es * 3^ no ' ' ' % u ncertain 

(1 not applicable included 
Cojment: here) . ^ 

Comments reported in body of report 



* * * * 



'81 



ujii«t? 



l-.Tiat changes, if any, would you reccrnmend be made in the trainuig 
Fornatlve evaluation Information rewrted orally to 



tjie developers and inoorporated in final changes made. 



f; * 
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SITUATION 



• The iollowinij areas or dimensions for comparisons among curricula have 
been sugj'c.stcd by a sub-connnitlco of the District Curriculum Committee. Now 
" -it is tiiae for each member of the Committee to vote for those categories of 
comparisons that are absolutely essential if the Committee is to do a really 
"export" job. On this and the next page, please vote. by checking (y^) those 
categories of compcUxsoTis that you feel ought to l^e^ised to compare different 
proposed curricula. 



S3LSCTSD CATEGORIES OF COMFAaiSONS 

pp-jCalegc-^'^r of cpmparisons POST 

5 niiiiiber of pages of student self- instructional materials 21 

. 90 » scope of the content covered 92 

\ hZ quality of the illustrations and graphics 29 



\ 95 \ intended results of instruction ; 99 

function of the teacher ijnplied in the materials 90 

'^Z * social ethnic or sexual bias in the materials 62 

83 ^^organization of the^content 81 

87 reading "difficulty level of the materials " 75 

role of the student implied in the materials (passive/active) 86 

• ' 25'; a^ailaMlity of teacher manuals • 38 

Y 53 ^teachingp^learning method implied or stated 

if3 size of^ student groups inJ'ierent to the materials ' 79 

presence of additional reading and study lists ^8 

52 special requirements for classroom staffing 87 

55 ' grouping of students required by m/iterials (age, ability, etc.) 66 

. if7 qUfdity of the witing in tlie cuiriculum ^1-^ 

57 necessity 'for any specialized equipment or hardivare 88 

- special subject matter competency required of tbachers using tlie matggials 





- 


post 


32 


physical arrangements £or use inherent to the materials 


73 


6^ 


cost per stmlcnt to purchase and maintain these materials 


96 


93 


Kind 01 student evaluation built m (exams, no exams, etc^.) 


90 


^^•9 


reusability of the materials , 


7h 


30 


comparisons of students built in (norms, class comparisons, etc.) 


57 


27 


availability of ahsv;er manual for any problems or tests 


34 


60 


infonnation on liow materials worked in classroom trials 


64 


^7 


quality and durability of the materials 




ilO 


convenience of tlie materials for student handling/storage 




88 


goals and objectives stated for the materials 


ipo 


^3 


consultant assistance provided in the "package" 


56 


38 


length of the complete curriculum in weeks or semesters 


78 


68 


educational philosophy implicit or stated 


86 


55 


ayaila' llity Vf tests for materials 


61 



Any comment: 



Signlfloant oo-nments rer)orted in body of the rer^ort 
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SITUATION 



A suh-conunutec of the District Curriculum Committee has submitted the 
following set of objectives for a new Spanish program for Grades 7-8, 

Spanish Progi;am, Grades 7-8 

All of the following objectives should be achieved by our students by tlie end 
of the eighth grade: 

1 . TJie student should be able to translate correctly any written or oral 
material described in the Eighth Grade Course of Study for Spanish, 

1. Mien speaking Spanish to others in the class, students should be 
using correct pronunciation and intonation* 

^ 3. Students should be able to demonstrate comprehension of wTitten 

passages read by the teadier by correctly answering questions (oral 
and. written) dealing with the content read, 

4. .Each student should have mastered a basic vocabulary of approximately 
1000 words. o£ Spanish and be able to define a random sample of those ^ 
words in Spanish. 

5. Students should be able to .read a simple story in Spanish and 
\ converse about it comfortably in Spanish while demonstrating 

appreciative understanding of the story read. 

. \ -k Ik it ^ 

- Given tlie achicvanents and behaviors represented most heavily in this 
set of objectives, please indicate by checking on the opposite page those 
classroom' procedures aM conditions which most logically should be followed 
by teacliors in the schoOjl district. ' . • 

First select the" most logica l choice (s) of teaching/ learning method and . 
then prckeed to indi catTybur qtlier clioices witliin the other categories^ „ 
listed". ■ • , ■ . 

Take time to carefully study the set of objectives given before marking 
your choice of teaching/ learning methods. 
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---PA RJ IAI. LLmNG-QF--0Fr-IONS-/a!OICES FOR SPAI'lISH INSTRljCTION > 

PREVIEW/EPILOGUE results in perceritages 
Tc ach Ini'./I -cn rh ing Methods > . " 

Icctvirc/Ucmonsiration 30/ 39 prograiiiiaed instruction 3 9/3 8 memorization .32/51 

laboratory/practical exporieKce8l/86 discussion/seminir 31/31 

rcsearch/syiitlicsis ^f/O discovery/inquiry I9/9 ' 

Teacher Function 

information purveyor 39/62 diaRnosticlan/prescriber? V66 contractor 
jresource person 61/55 fellow learner 9/4 no teacher function q/q 

Student Functi on J 

listener/follower 4 V7I fellow learner 29/ 9 self -instructor 3i^/l 6 
mutual planner/performer 68/56 pjrijnary planner/performer ^/j^ 

M'aFerials ' 

standard texts/workbooks 42/65 selected readings 3^/36 no materials o/Q / 
programmed materials 55/45educational games 29/25 A-V materials 47/52 
tcaclier/student prepared materials 55/30 specialized materials 36/3 0 

Physical Setting ^ ^ . * ^ 

single classroom with fixed furniture Vlp single classroom, movable 
* furniture 57/79 

- ^ open space. .classroom with moveable furniture *and walls 4Q/19 

library 25/1? study carrels 43/44other in-school location g/^ 

._ out of school/industry location lO/O 

Equipment (not materials) 

small hardware (T\f, talking typewriter, projector, recorder) 2£/8l 
large hardware (responder s> - terns with computer tie-in, machines) 27/1? 
specialized equipment (art, science, cooking, etc.)-i?/r. 
no equipment needs 8/8 

Bvaluation • " - . " ' 

teacher evaluation of students 2 9 /^'-2 Student self-evaluation 2^18 
bothjteacJier/student evaluate 82/65 evaluation by standard tests 25^/1?. 
mastory level evaluation j[j4/ 
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SITUATIQM 

It is difficult to talk about new instmictional prograins without 
considering costs. In this meeting of the District Curriculum Committee we 
want to do a' rough .cost analysis for the newly proposed Career Education 
Progran. The Super intui^cnt has indicated that $10,000 is available for this 
prognuii. We need to tclAhim how many classes of students (average size , 
32 students) could be seijVcd. by this program each semester. He has provided 
us with the fpllowing budget information which is used in thi.s District for 
calculating the annual budget estimates: 

Cost Figures for Our School District 

65% of any budget allocation goes into direct instructional costs 
(teachers., textbooks, professional support, clerical help) 

35% of. any budget goes to our fixed costs or overhead budget ^ 
(i.e. non-instructional costs) 

Of an Instructional Budget Agufe:' 

80% goes to teacher salaries 

10% go6s to professional support salaries 

5% goes to instructional materials (texts, etc.) 

5% goes to other (clerical/secretarial, etc.) 

•rhe ct iculuin materials for the proposed Career Education Prograni cost 
^$'10.00 per set., 

I. The average teacher salary is $10,080 for 36 weeks of instruction. 

* The average professional" support personnel salary is $7,920 for 3,6 week; 
of instruction. 

Clerical/secretarial help averages $600 per m.onth on a 12-month year. 
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The C ost Analysis Form (Our District Form 378) must be submitted with our 
answer to the two questauns at the bottom of the Fomu Please do whatever 
calculations (in round numbers) that are necessary to arrive at the answers 
requested by the Superintendent. 

COST ANALYSIS FORM (//378) 



1. Tentative Total Allocation..; $ 

(Obtain from Superintendent's Office) 

**Proposed Instmctional Budget 

Teacher Salaries $ • 

Professional Support Salaries $ . 

^:^Jylaterials ♦ ^ $ • 

^ Other Expenses. $ / 



*^*Proposed Non-instructional Budget Overhead Costs... $ 

(Nate: In order to secure final approval of this Budget the person, 
submitting (or committee) must answer the follov/ing two questions: 

1. If tills Budget is approved, how many sets of new curricular 
materials will be purchased? sets 

2. If this Budget is approved, how many weeks of teacher 
instmctional tijne will be charged to this budget? weeks 



Correct or partially correct (indicating understanding 
of concexDts involved) 

PREVIEW 21 i . * 

EPILOGUE 61 ^ 

Inco-rrect or left blank (indicating lack of understanding) 
Sunvnary " PREVIEW 6l % EPILOGUE 39 % 

}fow m;^ classes of students could be. served by this program each semester 
(ncr:nai teacher load, 5 classes and 1 prep)? 

classes for - ' semester (s) 



@ a total cost of $ 
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J)t«fMK' Ixlovv tJio critical churaccoristics of the four different types 
of learning goaJh stressed by the developers of this uiiit. 



Definition 



1, Pre-Statod Goals aocei:>table 36 ^ 



2 Pre-Stated Goald Indicators ■ 29 ^ 

53 



3 pre-3tated Curricular Obj.ectives 37 ^ 

62 , ,1 



Pre-Stated Instruotiohal Objectives 32 % 



61 



* * * A 



According to the developers, how does analyzing our basic assunptions 
about man and learning relate to designing "goad" instructional programs? 

Soiac 22 could givo « partial a;isv7er to this 



quea tion trhich is hardly eaphasizod in Ilodule on Analyzing 
Basic AGS-^nntigns, Others could fe'iVG reasonable answers to 
question but not in terms of the developers. 
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Desicning Instructional Programs 
an' Evaluation ' 

T. Bentley Edwards 

Richard W. Watkins and Gregory Otto asked mo to assist in the evaluati 
of the unit Designing Instructional Programs. To do so, I attended a five^ 
session course, Educational .Administration 870 Y1 given* in Room 234, 
Educational Building of the California State University of San Francisco, 
.August 5-9, 1974. ... 

Eight studeni^s attended, three v/oraen and five men. Two were man and 
wife and were from New York, he an elementary principal, and she an f 
elementary teacher. The other two women are employed as elementary 
teachers in a Los Angeles suhUrb. One man, employed by a nearby district 
has been responsible for coordinating work on instructional objective^ 
for his district. Another teaches automobile Mechanics in a community 
college. .The other two are practicing school men, with solid experience 
behind them. All the members of the class are mature educators, holding , 
enlightened opinions a^out educational issues, and, for the most part, 
able to articulate the reaj^ons for their beliefs. Furthermore, all were 
skilled group participants. With these students, the materials were highly 
successful* That is to say, they generated the course content, stimulated 
clarifying discussions, and seemed to include sufficient novelty for this 
group of students, mthoat. .causing Jhe frustrations that derive from not 
only carelessly prepared materials with their pusizling inconsistencies, 
but alse from materials that are too ambitious and include more concepts ' 
and generalizations than the students can keep in mini as they review 
their former experiences. To quote one^ of the men, "I can now ad^pt a 
fresh point of view in working v/ith my school boari on curriculum, & 
rev3,sions." * 




.Designing Programs 



^ There are five modules, each contained in a separate pamphlet. Each 
modul-e is intended for one of the three hour sessions. Members of the 
class worked away steadily. Some slovmess at the start, resulting from 
extensive discussion, was balanced by faster work during the later sessions 
when the participants made fewer comments. 

« 

At the first session three teams of three played the game, "Chalk Talk. 
In it, a team chooses a school subject at ^ poCrticular level and selects a 
series of appropriate teaching strategies, and desirable settings. Each 
choice is open to challenge by members of other teams, and success in 
meeting these challenges determines the score.^ Complicating side issues 
are introduced that add to the interest. The game was completed in about 
three hours and all participants fov^nd the jame interesting. The single 
choice from several alternatives required at each ^ step prompted discussion 
but reduced the .correspondence with an actual curriculum committee meeting. 
However, logic and consistency were used as- criteria as often as the 
personal experience and^ educational understandijig of the players, so that 
discussion of amiliar school questions was' more sharply focussed that the 
shop talk often heard v;hen groups of teachers congregate. Appeal to an 
* excellent, "Glossary," preserved amity on several occasions. 



A second\paule, previewed between^sfessions as /"homework", direated 
teams of three or more to state their assumptions about human nature, 
a view of the world, social order,, and human learning. The materials 
led them to perform this difficult task remarkably^ well* Teams were then 
asked to check the consistency of st^'tements concerning" the assumptions 
implic^it in. a parable. V/ith the teams* assumptiolis nqjr- expUcit,^ 
opportunity was now provided to review tTie dhoices made during the game 
"Chalk Talk. " . ' ^ ' * ' 
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Designing Proicrarna '* ' - 3 - 



The third module provided a useful categorization of goals going from 
the {general to* the specific* The discussion demonstrated sensitivity to 
the inverse relationship between freedom and specificity, and pliunped, - 
TTisely in m;^'- estimation, for "curricula^ objectives," rather than for the 
more specifiable "iijstructio^.al objectives." 

At this point, the "Curriculum Analyser," was introduced, causing the 
direction of the class to veer sharply from a reasonably tight consideration 
'of human variation to a precise, b^ restricted viewpoint. Instead of the 
excitement experienced from refining a program containing elements, at 
least, of his own thinking, the. participant was asked to use tools, admijttedly 
excellent, to examine. a program prepared exclusively by soikfeone else. 
In a 9ense, the excellent work of statin,*; assumptions was now thrown aside* 
Use of the Analyser was not emphasized by the coordinator, When he asked 
for reactions at the beginning of the fourth session there was no immediate 
responsa, althou{^ later one of the class members said it reminded hir. 
of v/orking through a report to the federal govemmont on a Title I project* 

..The fourth pamphlet, Qf 1i^6 pages, led us capably through an assessment 
of ;j^urric'ulum constraints. All participants found this instructive as 
well as highly infofesting. Reinforcement through the programed^ . 
instruction was most appropriate, to the n.? ture of the content. 

Pamphlet five each did by himself* It tries valiantly to salvage the 
work on selection of coals and statements of assumptions. Again,, it is 
less exciting to use the tools of assessment on foreign materials, than 
on materials Wiich one has shared in producing a d w ich therefore reflect, 
at least partially, one's om biases a:'!d assumptions. 
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Designing Pro^aras 



^ 4 ^ 



' 7ith these materials , the stuuents participating were able to sharpen 
and increase their curricular understanding to a remarkable extent • In 
the chort space of a single week, I sm^ild estimate that they mastered 
the content of more than a single collegeosurse, and this in an 
almost painless fasliion^ • We have here an excellent base for a fresh start 
in curriculum instruction, a base organised and planned for additional 
construction, * ' * ' 

Because the unit is intended to be "teacher-proof," the instructor 
(called a coordinator.) purposely remains aloof. This particular instructor 
perhaps for a variety of f^sons, achieved tlds goal only partiai^iy, and ' 
\70 77ere allowed glimpses of a friendly, delightful human being, as ^ell 
as of an ±nform(^d and profound thinker about educa.tional affairs. 
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.Attachment 4 



^ ^ Review by Dr. James Olivero. 

In the attached review, Dr. Olive ro refers to questions (or criteria) posed 
by tl^e Program staff. These questions are included on the next two pages. ^ 

There are occasional references to specific pages in the copy reviewed by 
Dr. Olivero, on discussion that is more relevant to possible revisions' than to 
assessment of the unit as a whole • These have been left in the report, rather 
than risk possible distortion of the report through editorial revision. 
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FAR WEST LABORATORY FOR EDUCATIONAL 
RESEARCH AND DEVELOPMENT . 

1855 FOLSOM ST.. SAN FRANCISCO, CALIFORNIA 94103 r - 
Questions Posed* for External Reviewers 



In your revio\/ and report, I v/ould encourage you to respond to 
the Kinds of quGStions you might ordinarily ask 1n review of 
training materials being considered for use by you in your 
school setting, or in' a training session you might be conducting * 
for others. In other v:ords, I am seeking your judgments about 
the unit, made in a way that you think is most realistic and 
meaningful to you as a user based on your own past experience 
with similar kinds 'of materials. I will, however, lis't below 
the questions we would lik^ you to- respond lo. Needless to say, 
I would be very discouraged if your questions and our-s-were not 
quite similar and bout equally comprehensive; but if they aren't, 
please feel free lo de^l with your questions. My questions follow: 

1. Do the goals and objectives, as stated in the unit, or 
as' inferred by you from the content, address an important need 
for scfiool staff, which if met would result in improving the 
effectiveness with which school staff perform in their jobs? 

2. Recognizing that therenvill be some relatively minor 
ravisions made before the unit is released, does the unit in 

'its present form and content seem to be well directed to the 
stated goals and objectives? ' * . \^ 

3. 'The following ques-tion? are more specific with respect \ 
to the content of each of the five modules. 'While they might 

be answered "yes" or "no," I hope you might amplify on your 
answe rsV| ^ 

MQ dulo 1 : Does "the pr^jccss of program design to i'/hich par- 
ticipants are introduced in 0\is modulo appear to be apDrapriate 
to the desian task, and ^if appfcopriate, does it seem cd^^ehensive 
enough? Will ^the j se of the gafhe as the vehicle for introducri^g 
participants to tfie process be seen by tiiem. to be relevant to tl\e 
task of designing instructional programs? 

X ' .Modu le 2:^ Is the process of analyzing assumptions important 
enougFTTifr the solution of problems of desigrting and impld;nenting 
• instructional] p rig rams / that this module should bo retained, if i,:.,5 
limitations require el imination of some of the content of the unit? 
^ Do the simulated devices- used in this module assist in the achieve- 
, meht of the objectives for the module? 



TELEPHONE (415) 565-3000 ^ > ' • / 



Module 3 : Does the use of Bloom and Krathv.ohl 's Taxonomy 
appear to be useful as a way of classifying goals and objectives, 
and does the process of relating goals and' objectives to in-struc- • 
tional program characteristics seem likely to improve the capability 
of designing or choosing an instructional program? Does the method 
of presentation seem effective? 

Module 4: . Do the^ devices presented for analyzing re-^ources' ' 
and constraints appear likely to be useful to school staff in pur- 
suing such an analysis? Do the simulations used to assist-staff ;* 
in using the devices seem good?- ^ 

Module 5 : Is the Curriculum Analyzer a tool" that ^schaol staff 
will find useful in their work of designing or choosing instructioijal 
t? programs? If so, what are some possible .use^ for it? Is the 'content 
of Module 5 adequate for learning.liow to' use the Analyzer, and for 
identifying needs for its" use? 

■ ' ' ' ■' / ' 

GT'ossary : Are "the definitions presented in the Glossary accur. o 

and consistent with accepted knov.'ledgeable use of th^ ternjs? Recog- 
niring that in so-'ie . cases coirmonly used terms have ,to be de-fihad in 
specific ways for a given training situation, even though -they hv/e a 
variety of definitions in daily usage,' are there definitionCin 4ho 
Gl ossary' that , you beTieve could be quite incon^ect if used in other 
situations? / " . . \ / , ^ 

4/ Is there additional information or materials t'lat you be- 
lieve v/o'uld simplify or improve Jie work tiuit the unit Coordinator 
, would have to do? Are there situat.ions qr kinds ^f Coordinators 
, where you might predict the unit would not v/ork, and if so, can you 

describe the limits on us^? (Wo know that tl\e unit can be used 
• effectively by- jion-Laboratory Coordinators, because it has been •done. 
What .we don't know, ^ind have neither time nor money to detennint^, are" 
the limits on effective use of the unit.) 

^ It the unit were to be used, but the staf-f ,ould not 

.devote the full fifteen hours judged^to be necessary, what- would' 
be .your reconniendations for .sections to/be elilninated, stated ap- 
proximately in priority order? (What .could be eliminated first 
, .what second, and so on.) • . _ 

6. ..'Do you believe that this unit would bo used in school 
staff training,, if it were avail a])lo at a co^t of $12-18 per' 
partictpoTit; plus a charge 'af\$2 0.00 for Coordinator's flatf. ^ils? 

7. 'If a school district sought your reconjneiida,«i-dn 'as a paid 
consultant for a-staff training pragrauj, and yQ{j- weriil^-sirtlsfied that - 
staff was clear en what theyV/anted to accpmplTsh iif a new bro^ramT— ^ 
would -you reco:ftnend th| use of this unit? Whai^tO't-hsr-tTai^iing pro- / 
cedures and materia.ls might ybu consider as aRernati ves to this one? ' 
How would you rank this unit relative to ti-ese other materials? 
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'"DESIGNIi^ INSTRUCTIONAL PROGRAM 



Prepared for RicTiard Watki'ns PreparsQ by James L. 01ivero-\ 

Educational Management Programs Nue.va^earninq Cefi^r 
F51A/ 1974 . ^. ^ HilM)orough,^Caliiornia: 




This program reruiew^has been^ prepared u^i^' J:he-criteria d^-^* 
■ ^termijied bj* Educati6nal Manigement 5>rogram Director Rich^i'd " t 
^^^^T\s, OtRer criteria .qged by the* fevidwer incluaed those o'f ' 
relevancy, feasibiMty and utility (operational definitions of 
th§se^ terms will be supplied later in the report). A ''miscella- 
neous" -section is included in 'the report covering other thoughts 
/which occwrred^ to the reader but- which did nfot match up with the • 
other criteria. . 

The '^Designing Instructional Programs " ;>publicat ion, "Curri- 
culum Analyzer, " was used to summarize and cpndense certain 
features of the curriculum; and the procedure oyt lined "on page IX S 
of the *publication was followed tg arrive at the' rating scores. ' ^ 

Each of the training bobklets was returned with the re- ' 

' viewer's comments. The booklets also contained' various comments 

^ • ^ • " ^ . ( 

The first section of the. report addresses the questions raised 

by the representative from the Far West Laboratory; the second 

section, io addip4.on to^thos^e above, addrfisse? /criteria prepared 

i ' ' / ' ' ^ ' ' ' " 

by the reviewer^; and the third section contains other comments 

^ » It 



.which seemed pertinent^. 



\\ \ " ^ • ^''i 4 ^ \ • « . . • . 

. '*■'' " INTRODUCTION > " . ^ r 

• . , ' V ^. . • . 

w ,aTTe* review -of .the, "Desigriiitg Iristructional ^orograms" mat^-^ ^ ^ 

\ . ' "^^^•^f ^"^^"^.^^ analysis of'f^ive module packets; a glossary^ of 



terras bopklet^ncJi^-a document entitled* "Curriculum Analyzer " ' ' i 

Tl^li^^sspciated -training materials, 'i.e., "Determining In^ 
.structi:6nal Pugposes" and J'Bvaluatxon for Program Ijnpr overrent*' 



V 



;were not; 4^onsi,dfered ^^7hen the "Designing Ins'tructional Programs" 

♦ . . - * . ^ \. 

' . * ^ • * .. ^ 

modules were 'rev.ifewed . The reviewer is biased/ positively, toward ' 
the use' of training' materials to assast with the man^ problems. 



faced >y educ'ators . . Most educator^, rieed new attitudes, skills, ' ttL ^ 

knowledge. These biases understandably influenced the a^alysiyS.. A 



MODULE l"^ AND, GLOSSARY ' ' 4 

The Goals and' objectives of the first module are stated . ] 

clearly, and the game^appears to be a useful tool t^o ac^complish . * .4 

the desired ends. ^>?hile I have not played the game, hav4 talked' ' ^ . | 



wit|i paople who hav^e, and ^hey have agreed that the topi. is ade- \ 
quate.- Cleatly, the many options^ cmentioned in the game, ass the 
teams progress up the ladder, shouU^ lie benefi.":ial the users. 
Bec^^use of t^he limited choices, however, tfee teams may not con»- 
sider .all of the possible options. ' • ' ' ^ | 

The j^ame illustrates nicely *how the complex parts fit into a 
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^ Whole. -Ev-an ip .the mini-ver?-ion of program designing, 'the user 
/ • ' • .... 

shouia be iil^le- to-see how the decisions .on one r*mg of the laddBr 

./have a l?e<^rlng on, s^ibsequent operations. Th^ gane, s?ould be very ^ 
helpful in, giving 'the participants a, obmrnon^ira-ne of refL-en<:e to 
q'iit at ehe.^r di £.f I'cultVask-: * ' . ' • ' ^ 

■/ To some individuals, the game will prob'alDly appear to be' ir- 
^relevant to the task af designiri.^ ins.tructie)nal .programs for "two 
■ ^rea/sons-: 'jl) Secause it" 'is .a ga-ne_ (a part o^ the descrj.pt^ ' , 
^ indicates the simula'tiqn is less than "real life," page^l?), and- 

(2)*, because ("he .directions' qo iiot communicate wh-at is reailv sud- 
^ ' ' • " . " ' , . . ' ■ ■ ■ ^ \^ 

posed to happen in the game (at le'ast, they' do not communica'te ' 

■« ' ■ ' .' ' ■ •> ■ 

this- to me).* j?or tHxs latter reason, it appears that a traine^ 

consu.ltant.woul«l^need' to accompany the package. If the. ^ame'is 
Expected to stand by itself ,. some add.itinoal developmental work ^' 
is. ^nec(Bssary • ^ /: " ^ * » * • - . . 

T;ie only real ^'n^rW^rn ''I . hav.-,-' ,ibo;it the gaW is that -yhich I 



havfe for all prograiD designing. Essentially, that concern. evoJ-Ves' 
arourtd the tendency to gocus on solutions to pri)blems "bf the'pastl 



Tnat IS, 



am Very concerned ^thae we^ ntight >e developing an ex- 
tremely ijiiportant' and pov/erful devi'ce which will assist people to 
do well v/hat they should not |De doing anyway. Perhaps the aseump-" 

tipns module helps *t;o cut iXicH the danger .on this • 
^ • • . . ^ ■ * \ . 

• * The,^ "rUng" of the ladder which,.! ' thought ^ was missing from the 
options ;had 't6.y^o with in^^servj-ce/edueation^ f or those expected to 



implement^ the prOi^Tam; Perhaps this wa'ss.' tak^nX for granted', ti'be-*- 
li.eVe;, however / that we Have Jea'rned enougfi abgat ^n^w curricula. ■ 
t<5 know they pannot be p^ed without ^eacher/s ""and administrators* ' 
learning new slCill-s, attitudes, aad kndwl'edqe, .. 

»^ " . - * » *^ .c/^ 

As a , design consideration, ^^I vjoul^ suggest Ifh^t \each partici- 
« — . ♦ • — . • * ^ , • ' * , » . . r 

pant^iYi the garfie have a set of, the .rules , a's' passlpg. a^single .book/ 

around to all players would seem to- be top t-^e-ionsumjing*, 

Ffankly., I foujid; that keeping ^ tracJ^.of twentTy-eight rules was ' 

more tBl^in I could handle hy ^simply redding gather than learninq . ^ 

' ?t ' • . . /. ^ ' * • ' • ^ 

the* rules by pl^aying* the game. \ * ' ^ V 

Someplace in the f i*r5t module booklet\ I jDelieve' it: would be 

a good idea to; include, some sort ot flow-qhart picture for the 

reader; The flow-chart would show Tiow the'modules were to. fit 

together. ^Without this' crutch it is' somewhat taifflcult to fdl^ow 

the step-by --step progression througt^ the tqtal ^ac^age/ ^ • ^ 1- 

/ • ' Obviously, the "Glossary" is an important part of the ^ame. 
Perhdps some^ mention of it should b^ made early so the players 
are^able to glahce thi^ough it. This would enable them to con- 
sider definitions of terms at the" same time they bec6me familiar 
With the game board and the game rules. It would be interesting 

to observe a pers^ introduciing the ganje to see how one individual 

^ ^ 7 ... • 
appro,ach <fio\x\^ match what is written in th4> booklet . * - 

Probably' some' additional introductor/ work in the ^booklet is 
necessary. ' l' would guess that on a random sampj^e of, ten principals, 
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game ^as--i : is'^fibw 'wrlc- 



7' opl^ three wWd be"a^re,.to iBtxoduce the 

.t>n. eor the ^ool-^to .obtain;tfte. p^tentic^ spme work/, 

\- needipr*in 'this ^arjja. -> A / • 

• .•Underltandablv,'. if^the game is t-o jDe Effective-. as* a toQl , 

' P^;^^icipVnts will need to,,^xndera.tand the defiriitions of terms 

\ .' ' ' *' '" « , • * 

in' the "Glossary." Perhaps'' a pre-t'e.st should .given 'to deter- ^ 

- • ' ' "r ^ ' 

. mine the "entry level" of the' learner, .enabling W ,to* skip what • 

he already knows. -(Parenthetically, I do. a wbrkshop caile^d "Inno- 
' , ' . • . f . * ' . 

vations in Bducafion"^ and 'always administer, a pre--test. Educators 

; ape amazingly unVnov/.ing ! ) "The definitions in the -"Glossary'.' are ' 
* . " . ♦■'<'■ • ' . . . ■, , ; 

• those, generally used-. (I 4iave made notes" iri^the booklet where -other 

* ■ . '■ • ' > ■ 

, terms are also applicable) .' •, ■ ' ' " 

t .. ■ 1 • . - • 

I would try to reinforce the' definitions in the "Glossary" •/ 

: ' . • ' ' 

with a sXide-tape presehtation offering visuals' -qf studerits, ; 

, teachers and so forth to illustrate the , definition/concept^ (I 

happen to. haVe"'a' ^lide-tape ^Jia-t gets at tt^NLs id^a, ifi you would ' 

care to see it.) Undoubtedly the. game will \be. only as 'powerful- 

as the concepts under^stood by,, the players.. I coiMd b^wrong % 

\ ••- • i • ) ^ ■ ' 

about th'.^"en-rry level" of educ.ators, but just tr/ a feample of ' 

fifteen to^ see how many know the dff'ferenc^s between wet and dry , 



carrels.,, and this might gi.ve^ Some feeling for wh^t little is 

really known. Perhaps' you ^ave^^already done the, .".entry level"' 

' . > ' "X- ■ , ' . 

asses'sment,'- and. in thiis qase, disregard the comments." \ 

a.- • ,' . * . \ . 'i . ' . - 

Two difficulties of the^gam^ become apparent wfien one thinks 

*■ V . ' ''\ ? ^ . X ^ 

of the . rules and af'the same time\hinks of the . forced" choice of ' 

. ■ ' I ^ \ ' ... 

.... Y . • \ . ' - . 



V 
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Jthe terms in .yie "Glossary .For exajnple , t»he term "Logical con- 

» * " * ' . * * ' 

'sistency*' i<s used frequently/ and tlier^e' is no*place I can find 
this* defiried . This, then, 'leaves "logical .consistency" up to the 

'vote of the players; and ignorance, even after discussion, coiiid 
tnean ^tbat s;^ecific -choicels were mandated-; that i^s, *the designer 

.may/h'ave been accurate in the first place but^ simply gpt oyt^ , 
voted. While, not knowing^ yoyr definiti^'on fop. /'logicafl consis- 
t^ency," it seems to me ^his voting procedvtre could be logically 

. inconsistent. . . 



\ <rn addition, because of ^Yie fprcexJ choiee . Con^j^' 6ne*dccepr\ 
table answer), the team of players. must make bnly o^e resjjjonse. 
if one- looks, for example, v at the "Intended Result of Instruction 
on page 3, there is a good possibility that at least two , intenc^ed 
results will, with logical consistency , be chbsen, particularly 
at a. time when affective skills seem to^be gairning *much greater ^ 
attention lin our schools. I can^nderstand th^e. forced-chores 



approach, V bot it may turn some people off to the point that* the^; 
becorae disenchanted With the tool, \ 

Finally, there are a number o-f^ dec is ion -making s^teps c^led 
* for in the game.\ I would hypothesize that while decisions can be 
made^by people in teams of two to'five members suggested' fpr the 



gante^ the problem becomes mor€^acute as s-^aff 'member^ increase : 

• ••^ * • « 

♦ <^ < • ^ 

for example* Ma^^ina Hi^h School in' Hjuntingt^on Beach,, California, 

' . ' ■ ■' ' ' • 

has 146 staff members. The key issueV^here is not- necessarily one 
' ^of design;, it. is one of problem-solving, and *corrtmunicat.iop^ That 

' . •• -e- ■ • X ' \ 



is, people cannpt get around to the design con*sideratit)ns; .until , 

•/ • • ' * • * 

they can do the iiiterpersonaJ. communication with each, other. Anv 

.- ■ ' ■ ' ' ' . \ 

ccxnsideration^ ahyplace for helping 'people learn problem-solv^^ng 

' ; ^ * ^ * • ^ : 

and cbnununica^ion sl<ills?, • \ \ , ' 

- . % , • . 

In summary, T believe the^^toolois worthwhile and potentially 
*very- useful. In my opinion,* it i§ nots yet fully developed if it ' 
•is to- stand by il^self . Please see th^ last? ^ctj^-on ofxthis r^- 
po^t along with comments in the module booklet for bther ^thoughts-. 



■V 



MODULE II" 



By aM means[/ Module II shopld be retained,. One pf the big- 
gest voids in -progji^am designing has to do with t^he inconsistencies 
between philosophic a^umptions and 'the acti/al activities ca;rried 
•;6ut in the classroom, Trife material used to ^accgmplish the objec- 
tives for Module II is excellent, from my point of view. (Frankly, 
if something in the total program had to be cut, I would be m^Drjs 
•inclined to cut the game.) Since each iiiodule is designed-^to ac-^/ 

complish different objectives, I really do not see. how' either 

\ 

could be cut and still maintain the integrity af thia program. 
The secondjntodule is especially well done. For example, the ob- . 
jectives are clear,, the activitias appear to»help pji^irtici pants 
achieve the objectives^ the strategies for learning -in. lude 
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independent study, diseu6,sion, and problem-solviiiig. ft3(5it:vc^nallx, * • 

•part, of the" program'is designed to give, the user specific fee^d- 

bhek on whether or not anticipated results have been acliieved. 

At fi^rst I was somewhat skep^tical of the four parabjlfes (and * 

thereih-. lifes a little ptoblem, i.e., getting^ the user to proceed " ... 

,^ . • ' - ■ ' ■ •• V./ 

past initial ^negat-ive reactions, so he^ can^ decide whether he doe^ 

ox does not: accept the .siiftT^latipn exercises^ as^ viable Ifc^ l .suspec^^ 

the personality of . the Coordinator wil], makfi^consideral^le ^differ r- 

ence.here; if the participants accept, the Cbordinafcor* as krii^^ledg^ ' 



/ 



abl^, the. parables will *fee accepted. If the use^s have little re- 

' . i ' ^ ^---'N- u ^ • • • • 

spect for the Coordinate!:, they will likely reject the "elementary 

^school" parables (win a-fev. . *iose a fewl) . ^ • • , - 

' 1^1 my estimation* there are l;twQ -voi^s. in the module: (1) 

techniques to learn about decision-making and problem-solving and 

(2), techniques to improve, interpersonal communication on the par't 



of the team members. .To omit th'ese or to take them for granted i^^ 

* ^ . . . ^ . ' ^ : . ^ 

naive. Perhaps t*he "Handbook" for -the Coordinator illustrates 
strategies for getting at' thes^ two factors. If not, I would rec- 

ommend same. (Incidentally, up to this juncture, I really have _ 

' , ' f 

not figured but who- the Coordinafco-r i^/ what training^ he/she has 

had, or whether or not there is a^agecific handbook or set. of pro- 

• M ' ^ 

cedures for this/person to followr I'wiTl focus on , this later, 
but I wanted , to br'ing it' up at this;"point because ^^odule l^l fre- 
quently mentions J:be Coordinator •) * ' , .v ^ • 

I am espc^ci^lly concerned at)out skills necessary to get ^at 
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a consensus-7 there are certain activities to .help people learn 



how'to do this, and I believe the package would-be enhanced if 
the CoordA.natcr knew how to do this and built in the option. 

Most of the terms (jargon) u^ed an the module should be 
Clear to the participant's* I mention this only because it is 
ano*ther . way to' turn people off. ♦ • ^ 

I belieyo the approach followed for getting at assumptions 
is good;» Frankly, it^ seems to nt5*"TKat before we can get^ at 
values for "others/* one must have in ntind what values one holds , 
.for oneself. It is because .of this that if I were doing the pro- 
gram, I would have star.ted with ^ some personal values clarification 



rather tha^ the 'concern ^or *'man." Only by knowing who I ain,/my 
values (real vs. stated) , my concerns for changing et cetera am I* 
"able to consider* the more globa*l issue. Perhaps this^s a small 
detail, but at le^st it is v/orth co'nsideifing. . ^\ 

I believe the interpretations at the conclusipn c^f the "Sug~ 
gested Respons^" section- beginning on page 57 arte very helj3ful. * 
They help -to establish closure on the important objectives in the 

module,, giving the reader comfort in knowing ^heAhe as^^tuned in 

^ ' - ^ ^ " ^ <• '] ' . . 

to. the task befor<^ going on to, the next 'section . This portJLOh is- 

really quite Well dojie. Also I liked th^ added* flexibility men- 

tioned at- th^ Bottom pf page 63. This is an improvement o\5^er the 

game,\vihere there is a forced choice ot.^optiorfs 

The "Prepared Instructional Program Outline" beginning on ^ 

t 

pa<^ 65 is outstanding, ^'here may Tpe a few^-opfeions not included*, 



.although they>do not, immediately, come. to mv -mindv. ^ Certainly, 
4the* device ^idSj the individual (or £eam) as 'the c^Jmplexity of tl^e 
^ is^ues^begins to evo3.ve. • Clearly, the systematic ' approach should 
X, be helpful; unfortunately, there is t<K> often the problem that 
' new designers fail to consider all the potential bugs in the sys- 
, tem until they* are too far 4hto the ball game and the t^sks become 
overwhelming: The. outline helps td avoid this matter. 

In conclusion, I am very excited about 'the - "Analyzing Basic 

Assumptions" module/ It is quite w^ll done.: 

» ' _ • " ** * * , 

' ■ 

— - 

. • ' ' \ ' 

I have' considerable difficulty with Modi le III, Frankly, I 

would either redupe it considerably or elimi^^ate- it a*ltgget^er. 

/ . ' ' ' • 

Undoubtedly, some of the/ i)oints made in the module are quite 

WorthwhiMe. With these given, let me suggest thoughts for possi- 

ble changes, to the material, . r 

First, objectives should not be written until the entry 

level of the learner is considered- One ^f the problems ^i^j^ J 

Popham's bank of educational objectives at UCLA .(and he is <^e of . 

*the first to- admit this) is the problem of people writing in 'to 

use the objectives and, m fact, incorporating them 1 ntt^S the cur- 

riculum without giving thougH;t; to actual student needs-. I dq-^not 

believe the statement m the preface that participants should^ ^s- 

♦ - • 

sume that a 'needs assessment' of educational values has already 

-.10- 
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bben* conducted " adequately provi-des the. caution sign. "Some se'c- ' 

. • ■ . ■ ^ '?\ , 
tion needs -to be 'developed on diagnosing stuuent' levels o'f- readi- 

ness, whether'^e stu.c3ents are adylts tJ^ childiren. . ' 

New topic: "Wj-le?^'! will men ti-6n' this point further at^ the 

• close of ^ this report.. .1 would to sugges^- here again that 'each 

booklet 'contaln-arnfcliart .early in- , the pages wh4ch illustrates how*- 

.the particular modulfe fits into the' package. This gives tfie use'r-^' 

■ . • '. - - ■ * . . 

a sense of from whente he/she has come and where the patV.is lead-- 

. • ■ ' « ' * / ■ .. 

mg. It IS difficult to get th6 Gest^alt", even after playing the 

game, without this road map. 

If individuals- agree ^^hat the objectives .for the module ar^ « 
relevant, the, content of the m9dule certainly is written in such 
a way that the user has a* c^?ance for meeting the. obje.qtives . 
fact, there may even be an overkill in one secti6n. For example,* 
if the usfer arrives at the answers to the questions on. tl^e "Goal - 
Answer Tally She^t," page 43, then the user is expected to con- 
tmue, essentially, with the same process o^ the next pages. I am 
not Certain tViis is nepe-ssary. It seems to me participant^ may 
learn more about the penguins than they really want to knov/. 
Cince people have demonstrated competency, there is no strong argu- 
ment to have them continue the same thing. ^ 

In this same realm, I am concerned* that the taxonomy exer- - 
cises may be more academic . than practical. If the u.^ers'were in 
a course at some njniversity; the d^^^cting anc3 breakmg-down-to- ^ 
the~lowest~common-denominatpr act ivitjj^ might be/ worthwhile. I 
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suspect, however, thai ^curricUlum designers iai:e moire concerned 
about getting on with 'the' t3sk .than engaging in \nental 'exercises . 

. }2v^n if the aBove position is n^t accepted philosophically, 
I wbulfl like^to fire one more hit of ammunition which may Ipe 
worth thinking about*: While it seems we might be helpful by 
'^J^reaking behaviors down into cognitive and •affective areas, I 
\vould 'aifcue th^t no action, fs totally ciffective or cognitive. ' In- 

deed, we neithef learn without feeling nor feel without ^earning., 

/ . 5 ^ ' — . 

< \ < * ' V 

I see -no plac^e xn the module where this* point xs xnade. Under- 

. ^ V . . ^ . ^ * 

standably^ I believe yov^ Should do this. * 

Another philosophical concern have is mentioned in ,page 19, 
i.e., getting parents, and other community members' inVolvfed. 1" . * 

• ' ^ ^ ' 

believe it is too** late at this * juncture to have them par^ticipate; 

rather, they should begin participating during the assumptions , 

♦ 

modules.- *If the goals and objectives-y are to etnerge from the as- 
sumptions * (philosophy) , then parents a;^<3 children must get into 
thp act earlier. ' ' • i - ' 

I douBt that a real iss.ue needs to *b^ made- about 

V 

tication of goals. Rather than work so hard on this^-^issue , I'be-^ 
lieve iS would be more helpful to accept global goal statements', . 

> . • ' . : ■ * ■ \ 

bi^ then to ask the qtiesfcion, "What, evidence is acceptable that 



the goal/objective has been achieved.?** This seenis to be a, far 
more reasonable issue, as it j>resumes that teacher^. are aware of 
entry levels of individual childrei^i, . T6o often when performance 
obj'ectjLves ate written,, people s^e the means as an end and fail 
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tex. really i^dentlfj^ those^ evidences that indicate .student growth^; . 
< . . : * ^ ^ . . . . ' • . .* 

.^I am wore concea^ned that people demonstrate competency than iJL- 

lustfate they have mastered ^thfe mechanics of writihg objective. 

Filially, the 'four items on page 49 seem somewhat esoteric; 

iarf^ ^x^ple, what ddes .-logically, developed ^nd internally .\ 

consistent" mean? How does one^ know y/heh this ha^ been achieved?^, 

..The^statentent "./.each class ^oiald^be purely descriptive and. not 

lm|>ly a system ojE^ hierarchy", doe^ not really say anything to me. 

PerJi*^s it. doe's «to- other pepjile. \ - 

' ^ .By- the way^^^in spite^of all my criticism with this module, « 

the chart of £3age 56' fs quite ^bod'. .1 expect- that ^some kinds of ' 

tools could be biiilt around the theme of the chart because it 
^ • ' ^ / ^ 

•shc^s hbw^to get\t the '*^vidence"' .issue.. 

You. may 'be interested, i'n additional comments' in the booklet . 

^ ^ ' » //^ ^ MODULE ,IV , r ' 

• * The devices pt^sented for 'analyzing resources and con- 
"'^'^ ' ' ^ "'"^ ' , 

s-trpiirits appear likely to be useful to school staff in pursuing 

such an^ analysis / p2?esuVingf they want 5r feel *the^ need for such 

an analysis. Frqinkly", having wqrked with h variety of ci^rric.ulum 

developm^t^ groups, I have not found them especially excited about 

or\ interested .dn making a dif^ect, fron*tal analysis of* the polir 

tical scene; rather, tTi§y have used such devices as the for^^e- 

^fi^ld analysis*, prptilent-solvirig procedure^ to identify co*nstraints 

• */ * ' « * " * • - 

and .j^oSsiblfe ways- for oveircoming same', , 4 ^ * 



- ■ ■. V 



It seems to me one major void omitted from conp-idera- 

* • 

' tion has to do with the "teacher training" element. -JWe._Jcnow 

enough froni history to know that few, if any, pfogjcams can be 

« 

installed in schools without appropriate teacher training. "This- * 
item is omitted both from tlig. discussion in the preface as well 
/ as H:he "Preliminary Assessment Foi^s." At the very least, I 
would add a paragraph indicating this factor has not beea included 
in the module even though in "real life" attention must- be given j 
to the matter- On page .4 th^re is a list of items which is use*- 
ful for doing pre-assessment . This is one of the points where T 
believe the teacher -training ractor Should be mentioned. ^ 

Even if the procedures as outlined in the exercise were 

* ^ 

followed, I have not been very many places where teachers had the 

/ « * ^ 

prerogative to trade off. the kinds of resources mentioned m the 
exerci4|. Building uj^this kind o^ false hope might be devastat- 
ing in tn^real-life situation. At least sOme word of caution 

should be made about this point; 

< 

The objectives on^^^^a[!ge^n6r are well stated, and having par- 
ticipated in the exercises, I believe the content teaches what 
the participant is supposed to learn. Again, I believe a chart 
would be helpful here to show how this section fits into the total 
scheme of| things (page 7) . One good part of the package is the 
help it gives to participants to fearn the common vocabulary. 
This helps them use the same terms as others when discussing cup- 
riculum design facets. / * 
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The exercise included in the section on pages 29 through 



43 ddes not seera^ to be particularly useful. This judgment re- ' 

fleets my own value system, but I da. not believe .busy educators 

,have the luxury to explore' the' esdteric and minute features of 

each part of the task. Rather than beirtg overly specific, most 

people are concerned about "gettin' on with i, t ' " This is simi- • 

lar to my bias about the taxonomies discussed earlier, i.,e., in- 

teresting but not especially useful. 

By the w^y, one, ad^lj^^i^ point on the "Preliminary " 

Assessment Form" seems pertinent. Parts' 3a and 3b illustrate a 

point called person-periods* I have n.ever heaxd of this term/ 

and f^did not see it <3efined. Does person-period Irefer to 

•teacher-period? If this is the case, why not explain it some- 

where? " ' * , ^ 

couple of places in the exercise on "trade-offs" the 
^ ' ' >, '» • ' ' 

suggestion was implied that money could be bor rowed, i-r Gin teacher 

to ^ . " *-w ----- 

salaries, doubt if in real life this Little matter works out > 
^ / ' ' " 

/this neatly.^ That is, with the single-salary schedule, the money 
to be spent on the teacher is a function of longevity of the' 
teacher andv not a -ipatter of cost-benefit or other methods to re- 

.deploy existing scarce resources. Th'dre is always the argument 
that just understanding the redeployment CQi^cept,is worth knowing, 
and that may have been the position the authors wer^ taking when 
the module was developed. I would certainly support this later 
position. 

\ 
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I have some additional philosophical concerns abou<Cthe 
editorial^izing beginning on page 122. For example, at the top of 
the page there is a point made that easy and open sehools lack - 
structure. In fact, I believe you will find that tl^e *'operi" 
schools (not necessarily the free schools) require more subtle 
structure than the more traditional schools^ Further in the next 
paragraph there is a suggestion that commup'ication helps to avoid 
distortions at either end o^ the philosophical spectrum. I would 
argue that this suggests a single rather than pluralistic philoso- 

♦ 'A 

phy. For me, it seems the latter concept should be supported if 
one really needs to editorialize in this section. Finally., there' 
is a d iscussion near 'the bottom of the, page about report cards 



^that is, perhaps, less than accurate. Many changes have been made 
4n the student-parent reporting systems in the last ten. years, and » 
I believe the example offered is more atypical than typical*. This 
part of the wriJte-up_jpakes me wonder whether the author has been 
out in the schools^ recently to know what things are hap'pening or 
whether the author is writing from an ivory-tower positions (1 

realise what I have said here ijs reasonably caustic, but X cannbt^ 

' *" « *. 

think of a faster way to turn off an audience* than to losfe credit-- 

ability. THe, writing comes close to doing'' this to. meVand I be- 

lieve the paclcage is potentially too J^elpful to purposely cause 

people to turn off. 

On page 142 tihere is an analysis, made .of a certain politi- 

/ „ . ' ■ - ^ . ^ " . ■ J / 

cal situation/as peome perceive it to exist. I would offer 'the 



^ 1 
" • -j 
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suggestibj} that -one could .'take the same basic data and have -them 

v'\ . . ' ' • ' • - ^ ■ ■. 

perceived differently by different people. Ask four different 
•\ 'I' ' « . . 

people th^ir opinions about the political structure of a- school ' 

1 '*'-./ ■ ' ■ 

system, and you invariably find, four different .perceptions/ i1c- 

t . ■ - r- ' • • ' 

respective of what tlie "true" picture might be. Even w^en a" 

' ■ " * ' - . ' " . ■ ' ' . * . 

teaqher knows the political' structure , there is little, <if any- 

I ' • # 

thing, a single individual can do about the situation* Therefore, 

ain not certain what the ekerc'ises do to enhance the package. 
Clearly ,i people ought to understand that political constraints ^ 
exist. This raight be handled in a 'paragraph. * ^ ' , 



In summary, while many Mparts of the "Assessing Resources^* 
^and Constraints" module are liel'pful, I would recommend that the 
total, package be reduced in magnitude aboift 50 per dent.' You 
may be interested in other commen'ts contained 'in the booklet. 

i _ - MODULE V and CURRICULUM ANALYZER ■ 

■■'6enepally^ spJaking, I beli.gve Module V and especially 
t^ie^ "Curriculum Analyzer" will be quite useful to school pefson- 
rtel.' It helps to synthesize the earlier elements of the program 
into an integrated whole. It is a very nice piece of work; as a 

I £>■■"., I 

matter of ^f act, the last section with just a little embellish- 
,'ment could pr©feab].y serve as the entire packagfet including the, 
lone on financial constraints. If I were using the program] I ^m 
certain I would focus most of the attention of the learners on 
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the contents of Module II and Module V. ' - • - "> 

. «^ _ . . 

' Let me g^t more specific with the crrb-ique: The pack- ' 
' . . • ■ * 4 ^ > ^- . 

age is- probab-ly useful, for teachers, but it seems 'to me" it could 

'] * 

also be helpful for' members of the community who want ' to' get into 
the curriculum- selection process but^ do not have the skills re--" 
.quired. Do you purposely waht to ignqjre t^e "communiey"" issue?"* 
Certainly the' objectives ^re well stated, as usual, and 
the "Curriculum Analyzer" .car^|?help a wide array of audiences ad- 
dress pertinent curriculum question^ .in- a systematic way. 1- do 

. have- sox^e qualms \nth Sne^of the objectives. The first objective 
on page 3 suggests that the commercially-prepared *desoript j^ve bro- 
chures 'are useful for analyzing curriculum pfferings. Experience. 

Steads m^'tb believe that this is ofie of the poorest sources. When 
those ^descriptions- are prepare.d by the publisher , the propaganda 
is sometimes leas than accurate. Perhaps 'sorne distinction should 
be wade alDout the appropriate place to locate information which 
pulls "together, ideas 'about curricula.* (Note ^should al^o be made 
that not all curriculum summaries are organized in a fashion' sudi 

• ' ^ . ♦ fir , 

• * ' • 

as that produced by the Far West Lab or by certain other agencie^. 
Perhaps the ^irst screening could be accomplished without ha>inq 
hairds -on experience.; subsequent efforts might be completed with \ 
iactuM'"Tnaterials r"'^'^'-- ^..^^^ • * * * 

^ A I^^believ^ it^^wpuld be helpful, too,, at thi? .point- to tell 
the users that the curriculum analyzer eitlj^r. can pr cannot be 
^ used to evsiluate currlculupi materials prepared by other clas'sroom 



^teathers. Most teachers who^lhavef Been around awhile and who have 



some moxy have figured -out a variety of ways to put together con- 
.bributions from, alternative* sources , making a short-of eclectic 
smorgasborg which turns out pre^ity weJLl. I can think of only two 
br -three source ^bogks whick'give any note at all that more than 
one approach is\possib:|.e /and that^'is wh^-Hihe publisher is try-- 



ing to sell I \ 

S ^ happ/^Tb see in this\ section some concern qiven to 

the idea of in-rseryice educatiiin. Bi^thex the consideration was 
ppitted in other modules ox j^t'^s'so casjue^y brought to light, 
that it w^s not noticed. I do b^l4e"via.,it is. an importajit matter 
that,.fteeds attention and believe aqme tliought ought to be given to 
^the lissue earlier modules , 'f.e. , the financial section.' ^ w 
I' thouqhty;the s:#iectiVn of the tv;o different types of : * 
reading ptpgr^ms wa.s quite good. It gave people a very different! 
option as to whi&t miight be selected. . (I do not know whether the 
EDL entr^ has been used at the third-grade, level ,r but •! doubt it; 
and unless they have something quite* nSw,. there is no way 1 can 
imagine it as b^^ing useful as a third-level reading program. I 
doubt that many other "people would thinks so either 1) 

I The one possible dif f.^^lty' with the select-ion ^of the 
EDL progijam has to dp,, again, with the credibility of %he*infor-^ 



matibn put out Jjy the* pub;Lisher* ^ I realize ±he ^lab' stalff members 
prepared the document that lis??* EDL as a reading program, but I 
doubt ;that they took the time to actually test the program at the 
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third -grade level and simpiy accepted the commant4^reaented by 



the published:. The British concept is useful not onlyf^s an op- 
tion to EDL but also because it gives ^most public school people 
an opportuni*ty to consider the politivcal climate, the program 



suggests it is legitimate to have children as rion-reader.s as late 



as the junior schools. There are prbbably few places in the 



United States at the present ^time vthat would accept, this concept, 
sofif the two different programs were considered together, all of 



the biases might point*, educationally and philosophically,. toWard 



the British concept Tlbut the political climate might negate thi/fe. 
In spite o~f thes^e comments, I wofuld stick v/ith the programs ,in,r 
eluded, * ^ • ' ^ ' 

- • \ \ : • " t 

The optional activities suggested at the conclusionNof 

t}^6 qnodule look as if they would be especially useful, particu^ 

.J 

larly if people were unable to make .up their minds about which pr 
greupn to^ accept. I pn, Lowevejf, not certain how the developer ar- 
rived at the value "7" as the indicator of what to agply to> the 
•different scales. You might want to mention someplace if this is 
just an arbitrary - value. . * . 



Let me move on to the^^Curricul^m An^alyzer." As far as 
I can tell, ''there are* no emissions in the curriculum characte*ris^r 



tics m'entioned.i 'Many guides such as this frequently omit two 
categories, one ^as to dQ H^^^"^ aipouitt of preparation time 

necessa-ry for a t^eacher to -use. cxirricuM and the oth'er has t© do 



116i ^ 



with the special skills', (including, in-service education) needed. 
I was. pleasantly surprised to find that both of thes'e" areas were, 
indued in thevanalyzer .'■ ' • , ' 

As you can -see, although I did not have a "stahdarc^i" 
..4,^^ ^cv|rriculum against . whiph to assess the different modules .that 
ha^^-tJfe-e'rr'put together, in the training package, l/did '"rate the 
lab. program- 127 out of a possible score, o-fs 196 . Actually, L see 
the 196^ score as being important only if one accepts the value 
of "7" as relevant for each characteristic .; .and I db not.' Per- 
haps a better (or a*t least a different) way to use the tool is 

• \ 

to take a look Sit the difference between the r'ati-ng score and the 

' I - i ' • ' ■ ■ • ' 

wfeightVihat a person gives each of the characteristics. If 'one's 

expectations'" are not ve^ry high for a given characteristic, then 

the difference does not ^become al^L that important.. There were 

times, for example^ when I rated an item highe/ than the .weight • 

value given later. ^ For thos^e whe^re the weighted value was higher 

than^the rated value, I would have 'cause for concern, ancf this con- 

^cern is probably mqre realistic than the difference between 127 and 



196-. , . . ^, 



Please note page ],7-<in Module V. I really believe this 
is an important :^ea, but I saw nowhere ^in the design of the 
package where ' this "was possible, i know' there, is much- to 'o with 
teachers and* administrators., but if alterhati^ves (different -phil- 
osophies) are goin% to be realistic, parents need to learn the 
skills too. Perhap^when 'we take some of the hocus-pocus out of 



-21- 



ERIC * ^ ^ 117 • \ , 



.education; we ■^ill have» grdater strength with the general public. 

The "Curriculum Analyzer," I believe, has gr^at poten- 
.tial "for tshis. By 'the v>ay, I believe the suggestions indicated 
that if the characteristic was' rated "0," then it should not be 

. . ;' - ' ' -.r ■ 

multiplied , by a weighted value. It seems to me tha't ther'e is al- 
ways a good 'possibility that something might be rated "0" and be * 
very -iipporta^t to the potential user. Perhaps the rating.' scale' 

# ♦ 

should go frofh to +7 or some other . such arrangement, Jbo show 
that an item miqht have a high weight but low or zero rating. I 
can even envision that what is available may be worse thap having 

'no information i at all, potentially a negative value* Tlie current 
scale does not really permit thi,s> ' 

As can be seen from the assessment^ there is a good pos- 
sibility' that I would use the materials if I ' had^ the opportunity, . 

^*but not because the congruepce between the potential point^* value 

and the value I came up v;ith was particularly gooc3. , The i selection 

would he made because of the relative few cases in which the value 

I applied in my owh judgment v/as higher thg,n what t believed the 

* ? « 

modules had ^to offer. * 



There are more commen.ts for review contain^^d in the book-* 



lets. ♦ ' J ' ' V J 

OTHER RESPONSES • ^: * - 

The folloCs/i'ng responses aj-e offered in d^r^ct ans^r t*p 



' . > -'^ «' ■ / ■ - - ■ • . * 

/ ' * - . ' \ * . * . ' ^ ' 

the questions ennmerat-ed in the letter ^at'ed ^August .29, 1974, 

excluding question 3, which I believe, has already been answered • 

* ► » * • * * 

The goals and objectives definiterly address an im- 
portant ne^a^^for school people 'a3 well' as •merabers of the commu- 
* * '\ >^ 

^ nity whp-are more and more being called upon to participate in^ 

» • . • - ./ ' 

educational 'decision-making. One o'f the reasons I believe the 

/pScKagd i s'^gooa'Tsf tlia^^ art^J^thtrig ^1^^ %S 

^ complete ajs the materials which have developed. Certainly, 

there are two argumjants related *tp this point: (1) The package- 

may contain 'more about some aspects than people really want to 

' ^ know, and on the otlner hand, (2) people need to b(? willing to xn- 

vest somfe time in learning how to dd^ a job correctly when t^e fis- 

*' c^l and human stakes are' so high. I would opt for the latter po- ^ 

^^^^^^^i^on, although there are a couple of places in the materials 

where anyone could debate the relevancy, of the content; but this 

, * same argument can be made on any number of curricular programs. . 

What turns out to be one person *e trivia is often top priority 

^, for a;iother person; hence, the reason to leave the context fairly 

integrated . * • » k^ ' 

If T' had the materials available to me, I believe 1 could:. 
: ^' " . ' - • 

^ ' use all parts with the participants, except the game, and not 

^ feel that * the experience would be embarrassing to them or to jiie. 
Naturally, if I wor'e to use the materials*, I would attempt to^de- 

^ velop somejadditional^ support materials to augment the modules; 

» - * * 

have"^ indicated these either in .the preceding paragraphs or in the 
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' columns in the module, booklets/, • . . 

I - r;- ~ — . 

/ -2. "^In short, there is no question in my mind thdt an ^ 
indivicjual willing to^invest at least some small time aad thought 
(20 hours, I gue^s!pQpuld achieve the objectives as stated in the 
materiars. There are some, areas that/, hopefully, will be modi « 

..,fi^;3.,...b]Ul; evea vif i.t ,were to stand as is, the produ'ct could be 

/ 

useful in almost- any school in any district, presuming two things: 
(1) The leader in the district or sghool is knowledgable about . 
the package and can convince others that the process will get 
them where they ought/ to be going, and (2) the interpersbnal com- 
munication between the people involved in positive to the- point 
that decisions can be mad^. What I^am trying to say is the .module 
.package 1^5 essential but not sufficient to pull of ,the desir*ed end 
results • \ I * 

Moreot^er, I .can See no easy way of dealing with this lat- 
ter problem, but it would^ seeiri reasonable for an outride Coordi- 
nator to be av)are of the potential difficulty and then to be ablq * 
to diagnose and treat the matter re:).ata<v,eLy- sc^on, should, the 
situation arise. ^If I were a Coordinator, I would start off with 
a few warm-up* communication exercises to help individuals get 
about their business/ ' ^"^"j: 

♦3..» Already completed. s J 

4* The only major modifications I woi:i/id make are two: 
a. ^ A handb'ook for the Goord j,nato'r t had the' 
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impression that there must be -o^e- someplace, but I did noi h^ve 
one when reading the other ^naterials. I am not certain what 

, Plans.j:he unit. coor-dinator;has for training other people to"^ use 
• t^he package. This is >inore of a dissemination issue, and I will 
not delve furthe'r into It at this point except to sly. if ^ou Want 
to have quality control, it would seem wis^, to establi^ some 

• sort of minimum competency levels for the persons serving as Co- ' 
ordmators. . 

If you do. not. have fiscal resourc-fes to^ determine mini^ 
,mal performan.ce leyels of Coordinator^ you must use your own ' 
professional judgment'to ch4ck out- cjeiteria. -When • you' dissemi- ' 

- »■ . 7 

nate materials, you may want to indicate someplace that for a 
small fee, you will also offer a consultant from a list of ap- 
proved, people who can' make certain that those using the materials 
reach expected achievement outcomes'. 

5. Rather than eliminating sections, 'it seems to me you 
.should find ways to reduce the time investment of personnel by ^ 
permitting them to move. ahead when expectations have been \net . " • ' 
There -are a. few places in the materials where tl^ere is overkill 
an^ t;his is unnecesiary . Some pe^ople- Witl^ reasonable experie'nce 
will tune jout -when -they believe they are investing their time and 
talent on sdfnething they alreadj^ know. (By the ^?way;^ 'there are ^ 
lot of people who' think they know things'but really do not, a^'d " ^ 
about the only way to convince these, people is via some sort of 
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dembnstrable test ... and -even then a 'lively debate is often pos- 
* ' • . 

sibie.) I noticed no s.uch proficien'cy options in the materials. 

If I absolutely had to prioritize the list*of modules, I 

would first keep the fi^th module and "Curriculum Analyzer.'* 

Then I would keep Module II- Next would come Module III, then IV, 
• • , ' - " , 

and Is^st vjtpuld ^'e Module I, the game*^ Perjiaps 1 would feel morp 

positive abojiit the 'game \f 1 were to participate in playing it. 

Others, have told me it^s'fun. Prom a. ^ales point: of view, it 

might offer just the right gimmiirk (using the term loosely) to^ 

■ . 'V ^ - ' *>■•._ 

help make t^ie *package "gp'- " - ■ / - \ 

^ ' * ■ • ' '■- . . ' / • . 

6- I do belie\ye the price -tag yp'u ^have put on the pack- 
age is reasonable- If a school district is going to make thd 
kinds of seri;;ous decisions called for. tfcay should expect to in- 
vest initial money inta people ai.d mc^terials* With .the Federal 

t • { ' ' o ^ ' 

^funds available through' different tit/les, m Califprnla' through 
- ^ . . ^ . 

3,3 leg^-sl^tion,^ et cetera, thar^e se^ms to be a' realization that ^ 
if programs are to work, *teachers need tjhe in-service to do tbem 
v;ell. The training .package, it can* be argued, is a stafi^ develop- 
ment option. iEven if teachers -in a given district were not.in- 
volved in^ the selection of curricular materialls (a;nd they usually 
are), ther'e is plenty of meat in the modules for /all educators, 
if they will . takS' the time to consider what is available. 
' ' . Colleges and universities* are oft^ slow to change, but 
It would 'seem that any curriculum and instruction people and/or*^ 

^ ( 

' .. -26;- • - * 
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I ^ , ■ the cari§9dlta33ts* at ^tHe California County Offices o£ Education 

*wdufd be logical agents to implement, the program. Incidenta^y, 
r/ the j^egional education service centers in Tex^s and the B.O^C^E.S • 

S ' , • . - - . . ^> ^ 

I ' If " in^ Nev/ Y.ork 'are alwajCs looking. for pack'ages such as this.' Texas 
. . i^=^eed^ now has a^law which r^quixes that teachers hav^ten days 
I ' ^ per year of s-taff development . ' ^ * . 

*' 7. "IJ^ definitely would .recoipmend, use of the materials 

* ^ and would use them myself -if thay were available. As I h^^ in- 

^ dicateA earlier, I have a fisv; different ways to do .the "assump- 

tions^' package and believe ,the personal values and institutional ' 

values topis use are more powerful than tha approach taken in 

Jlodule II, but this probably* is my bwh bias. In any case, I cer- 

itainiy agree that the a^ialysis of the philosophy of the school^ 

jnfust be the 'first , step, and there ^re many" Wjays. to do' this. ' 

Moreov.fer, we^'ll^ueva) would be willing^ to hel,p*aissemi- • 
. , . i - ■ ■ . . • . \ 

nate the modules through our modest Learning Center, although ijc 

^ I weye you, I would consid^er PDK and ACSA fpr potential , dis^mi - 

^/iiqjtion xn^that order. 'You may have better contacts than O-r but 

I ^ou3^d start with,T<eith Ro'se and Art Thayer^ respecti/^ely . 



' ' ' The oijly other materials and jproce.dtires I wou^Jra use, as 

H*\ ^ *^ "... 
' . alternati'V^^o what, has been pClt together, 'are those I'^have 

t . / * . , ' , / ' ' 

1^ developed Vjiyself ; unfortunately, ..fo^c^use, I have nev^r packaged^ 

* ' / the \materi^als, th^y are- Usually p\it into "and taken put of a work-^ 

* , .-xShop depending uport .many fac tors r that 'are not related to very. ' 



much of anything, except me. Some of the arguments for both^ — 

choices are obvious. Undoubtedly, thfe program is well done 

qet it out "to the people who can use it. 



PERSONAL CRITERIA - v 

"When I look at different curricula, there are usually ^ 
four umbrella areas corvsidexed: Aims, Content, Methods, Bvalua- 
tion. The specifics of these are quite well contained in the 
"Curriculum Analyzer/' although they are called "q^urriculum char-- 
acteristics . I saw no major weaknesses in the materials in any 
of the four umbrella areas • What suggestipns I would make have 
been ecnVeyed earlier. 

What I would like to do in summary^ then, is to offer 
some thoughts about the program which came to me as the modules 



Were reviewed, ^pt^se are rather aside comments and are offered 
iust for your *own use* - . *^ ^ ^ 

1. You *might want to sajr a few more words in the int^o- 
duction about the authors, esjpecially those parts wl^ich lend 
cre{2^w:ability to the work they have bfeen .doing in the^odailes ^ 

y 

With all due respect intended, I have not heard of any of. the • 
people other t,hwin Otto. I'Then Glenn Nimnicht did things at the 
Lab^ people knew they were going to be great because they had his 
narfie on them. Toa few^ peogile in the field think highly of labs, 
and I would do anythihg I could to Igt^hem know there ar^ good. 



people empJ.oyed there trying to make tunings better for kids and 
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edycators* 



2. If some'thing is not going ta be done with the biblio- 
graphy (and because it is not annotated) , why not just leave it 
jDut? Again, this cuts dov;n the size of the package and makes 
every page count, , ' 

3.. In addition to* the t;wo items I mentipned ecirlier in 
th^ report,- I would also give serJLous thought to the r-^ssibility 
of including conc^sus'-making anWor problem-splving tecHniques 
in the package. These are not^all tha.t hard to do; if the Coor- 
^ dinator rece'^^s special train'^ing, you may want to build the 
techniques inCo' the handbook he/she' uses . . 



4. Overall., I thought the wi;iting was well done. Some- 
one had 4made a careful effort to avoid the overuse of jargon. (I 
took the liberty of marking a couple '^f places where it seemed 
the writing was unclear.)'' 



5^ The materials as* they are designed accomplish the, 
• / - , 

goals and objectives* I must say, however, thjb format for the 
program is anything but exciting. I do not know who is workijig 
QjTlSiis part, but this matter could definitely use some attention, 
perhaps a systems chart would be beneficial, or 'even some iazzy 
ear too'ta characters.' Enough said; I expect something is already 



"29" 



1 



being done with this. 

6. When and if the program is completed, r would try to 
db it in such a way that you have two major components: (1> t!liose 
items of written materials and audio-visual materials t}iat are \ 
nonconsumable and (2), those items'that are' consumable* People ix\ 
the field could continu^^to order those' parts which are' used. I 
believe this can all be done and still keep the packaging costs 
within $50. This is not an unreasonable price to ask for what 
the peopls are getting. , 

IN RETROSPECT 

I enjoyed the oppoiltunity of reading and reviewi.ng the 
curriculum design materials. My interest, when critiquing the 
documents, was to be constructively critical. For certain,^ some 
reviewers might suggest throwing out the "entire project; my 
biases are much different. V7hile the package completed to date 
is less than perfect, it appears to offer reasonable help to 
people in^ the field who desperately need it. . , 

.It would be possible to continue further development of 

\ ' ' 

•the package. This is a matter which the Lab will needv to decide 
(as anij^^iabi'igust ) , based upon the availability of scarse resources- 
From my point of view, the Model T is ready and should be gettmq 
out where it can be, used. 

Possibly there may be some aredrs of the report which are 

^ K ' 
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unclear; sho^ tlii^; be the case, please /drop- me a li^e or*call 
on the phone. Should it be helpful for me/ to ^xplain in person, 
any of the poipts I have made in writing, please let me know, as 
arrangements can be made for this* - - ^ , ' 
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